DOCUMENT RESUME 

ED 374 634 EC 303 359 



AUTHOR 
TITLE 



PUB DATE 
NOTE 

PUB TYPE 



Feldt , Glenda D. 

Improving Delivery of Vocational Evaluation Services 
for Secondary Special Needs Students in Norfolk 
Public Schools . 
Aug 93 

172p.; Ed.D. Research Project Report, Nova 
University. Some appendices c on tain small print. 
Dissertations/Theses - Undetermined (040) 



EDRS PRICE MF01/PC07 Plus Postage. 

DESCRIPTORS -Delivery Systems; Disabilities; '"'Economically 

Disadvantaged; *Educat i onal ly Disadvantaged; 

Evaluat i on Methods; ^Limited Engl i sh Speak ing ; 

Models; Program Devslopment; Secondary Education; 

Secondary School Students ; Special Needs Students ; 

Student Evaluation; ^Vocational Evaluation 
IDENTIFIERS *Norfolk City Schools VA 



ABSTRACT 

When legislation extended vocational assessment 
services to limited English proficient and economi cal ly/educat i onal ly 
disadvantaged students, which comprise 72 percent of grade 8~12 
students in Norfolk (Virginia) Public Schools, innovations were 
needed to offer expanded services without additional personnel. A 
multiphase model of vocational assessment for special populations was 
developed, based on interviews with leaders in the field of 
vocational evaluation, a statewide survey of vocational evaluation 
centers, and a review of the literature. The Management Information 
Systems office tabulated information on students to identify those 
who met federal and state definitions of special populations. The 
middle school guidance and testing personnel added the Differential 
Aptitude Test to their testing program. Thus, the practicum resulted 
in all eighth grade students receiving a vocational aptitude and 
interest test, students with disabilities receiving streamlined 
vocational evaluation services, and fewer students needing a 
compr eh ens i ve vocational evaluation. The model has been 
cost-effective and complies with regulations. Appendixes provide 
survey forms, survey data, and administrative documents. (Contains 
approximately 60 references.) (JDD) 
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Abstract 



Improving Delivery of Vocational Evaluation Services for 
Secondary Special Needs Students in Norfolk Public Schools 

In Norfolk (Virginia) Public Schools, vocational assessment 
was available to disabled students since 1980. However 
legislation had extended this service to limited-English- 
proficient and disadvantaged (economically and academically) 
students, 72% of Norfolk students in Grades 8-12. Without 
funds to add personnel, innovations were needed to offer 
this assessment to the expanded population while continuing 
to offer evaluation for disabled students. 

This report describes the development and implementation of 
new methods and procedures for evaluating the vocational 
potential of secondary special-needs students. The 
preinterventiu.. vocational evaluation model required 
students to be pulled out of school for 3 to 4 days, travel 
to another site, and complete psychometric testing and work 
sampling. Under that method, each evaluator served 3 
students per week, or about 190 students (of 1200 eligible 
special education students) per year. Services were not 
available to the 7,000 disadvantaged students. The problem 
was: how to provide students with better access to 
vocational assessment services. 

In this practicum a multiphase model of vocational 
assessment for special populations was implemented. The 
interventions included utilizing existing personnel, 
collection of data, comparison to other school districts' 
evaluation programs, development of a new model of assessing 
handicapped students, development of a method for 
identification of disadvantaged students, and creation of 
assessment options for disadvantaged students. The new 
methods were conducted in the home school and utilized 
existing data from individual records as indicators of 
students' vocational potential. 

The result was that all eighth-grade students received a 
vocational aptitude and interest test, a more streamlined 
model was utilized for disabled students, and fewer students 
needed a comprehensive vocational evaluation. This model 
has been cost-effective, provides compliance with the 
regulations, and lends itself to replication in other school 
divisions . 
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Chapter 1 
Problem and Probleia Background 



Statement and Primary Evidence of the Problem 

A study completed in December 1991 (Feldt, 1992) , 
revealed that only 14% of secondary special education 
students in Norfolk Public Schools (NPS) began a vocational 
evaluation each year, that only 71% of those who began the 
vocational evaluation process actually completed it, and that 
only about 2 0% of special education students eligible for the 
vocational evaluation were referred. In Norfolk Public 
Schools no process existed for vocationally assessing other 
special populations (disadvantaged and limited-English- 
proficient students) . 

Kochhar and Barnes (1992) paraphrased wording of Public 
Law 101-392, The Carl Perkins 1 Vocational and Applied 
Technology Act (1990) : 

Section 118 (special populations assurances) and Section 
235 (Use of Funds) requires each organization or agency 
receiving funds to provide assurances that it will: 

1. assess the special needs of students 
participating in programs using Perkins funds, with 
respect to their successful completion of vocational 
education programs in the most integrated setting 
possible; 

2. provide certain guidance, counseling, and career 
development activities;. . . . (pp. 9-19) 

This means that all special-needs students who enter 

vocational education programs should have a vocational 

evaluation or vocational assessment. As Cobb and Larkin 
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(1985) indicated, this would imply that the "vocational 
evaluation process was completed and that recommendations 11 for 
successful vocational program completion have been made (pp. 
1-14) . 

Secondary special education students include those 
students in Grades 9-12 who have been identified as eligible 
to receive special education services due to various levels of 
mental retardation, learning disability, physical disability, 
visual, hearing or speech impairment, severe emotional 
disability, or other health impairments. 

Other students making up the population noted as special 
needs are those who are economically disadvantaged, those who 
are two or more grade levels behind the class with which they 
started school, those who have a low grade-point average, and 
those who scored in the lowest quartile on standardized 
testing. Obviously a great deal of overlap would be expected 
as students who were identified under one of these criteria 
would appear again in others. 

An operational vocational education plan for serving 
handicapped and disadvantaged students, including those with 
limited English proficiency, will help to direct students into 
three programmatic options, according to the Virginia Council 
on Vocational Education (1991). Their annual report showed 
that: 

Students are mainstreamecl into regular programs and 
expected to achieve competencies as any other student; 
mainstreamed, . . with special supportive services from 
additional resource personnel or equipment may be modified; 
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or placed in a specially designed program. Enrollments 
in 1989-1990 show that ninety percent of the 
disadvantaged and handicapped students (in Virginia) were 
mainstreamed in regular vocational programs, (p. 3) 

T v * challenge of keeping disadvantaged students in school 
had bee* a rriority in Norfolk Public Schools for the previous 
10 yea* , As Neubert (1990> asserted: ''Strategies to ensure 
that urL n special needs students have equal access to quality 
vocational programs must be foremost in the minds of regular, 
vocational, and special educators" (p. 2). A summer retrieval 
program, peer counseling, school-within-a-school , special 
counseling services, mentoring, and summer youth employment 
had contributed to a significant improvement in retention 
rate. However, vocational assessment services had not been 
developed and were not offered to this population. 

Drafters of the Perkins' regulations recognized the need 
for special services for disadvantaged students, as had 
previously been done for handicapped students, by grasping 
demographic trends and utilizing related research. 
Demographic projections had shown that by the year 2000, 
one-third of the United States school population would be 
nonwhite with African-Americans constituting the largest 
nonwhite group (Hodgkinson, 1985) . NPS was ahead of this 
trend at the time of this project, with a 68% nonwhite student 
enrollment. 

A challenge to educators serving large minority student 
groups was that as many as 40% came from families who met the 
definition of poverty (Levin, 1985; Pallas, Natriello, & 



McDill, 1989). Research by the William T. Grant Foundation 
(1988) has shown that risk factors associated with persistent 
poverty include school failure, dependence upon public 
assistance, marginal participation in the workforce, and 
female-headed households. Students struggling to break out of 
this cycle of poverty and unemployment are recognized as truly 
disadvantaged individuals (Neubert, 1990) . Clearly, 
providing services to these truly disadvantaged students will 
require alternative strategies for successful vocational, 
employment, and postsecondary experiences (William T. Grant 
Foundation, 1988). These strategies, including vocational 
assessment, are targeted in the xkins 1 Act and in Title II 
of The Family Support Act (Public Law 100-485) . 

Despite interventions through the introductory practicum 
to upgrade teachers 1 involvement in the process of obtaining 
the evaluations and efforts to inform students of the purpose 
of the vocational evaluations, completion rates had not 
improved satisfactorily, going down from 33% to 29% (Feldt, 
1992) . Referral rates and alternative methods of conducting 
the vocational assessment were not previously addressed. 

Students (n=10) , two from each of the five NPS high 
schools, who were interviewed after refusing to participate in 
a formal vocational evaluation, expressed an unwillingness to 
participate in a 4-day pull-out vocational assessment 
program, which was the model for vocational evaluation in 
Virginia (Scott, 1991; Scott & Prezioso, 1986). That 
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model consisted of (a) a formal written referral process; (b) 
the transmittal of copies of each student's psychological, 
educational, medical, social, and attendance reports to the 
assessment center; (c) scheduling, transporting to a different 
school site where the assessment center was housed; and (d) 
then completion of work samples, psychometric testing, and 
some situational assessments. Because interventions to 
improve the individual student completion rate had no 
significant effect, the focus of this project was on reviewing 
the assessment model, to determine its efficacy in the 
changing educational environment and to develop alternative 
methods and/or a new model. The project time line and 
implementation strategies are shown in Appendix A. 

This project w^s originally proposed to focus only on 
improving the delivery of vocational evaluation services to 
handicapped students. Based upon initial work and further 
interpretation of Public Law 101-392 (1990) , The Carl Perkins 1 
Vocational and Applied Technology Education Act, the project 
interventions evolved to include students who were 
disadvantaged and limited-English-proficient (LEP) , too. This 
population must also be served according to the regulations 
but had not been previously vocationally assessed in Norfolk 
Public Schools. The underlying premise that students who are 
disadvantaged will need additional services in order to 
succeed in vocational education is the same as that for 
disabled students. 

5 



13 



Background 

The Carl Perkins 1 Vocational and Applied Technology Act, 
(Public Law 101-392, 1990) placed strong emphasis on providing 
handicapped and disadvantaged individuals with equal access to 
the full range of vocational programs available to all 
individuals and stipulated that vocational programs and 
activities for handicapped individuals would be provided in 
the least restrictive environment. This act required that 
each handicapped and disadvantaged individual who enrolled in 
a vocational program should receive an assessment of 
interests, abilities, and special needs related to that 
program and special services designed to meet these identified 
needs. Note that "the word x aptitudes 1 does not even appear 
in the mandate" (Cobb, 1985, p. 4), but assessment of 
aptitudes continued to be the basis for vocational evaluation 
services up until the time of this project. 

The intent of the legislation and the requirement of 

vocational assessment were to open doors to these individuals 

so that they may obtain marketable skills. Greenan and 

Sitlington (1987) summarized the overall problem: 

The ever-increasing academic requirements of vocational 
programs and the occupations for which these programs are 
preparing students, however, serve as a major barrier to 
the successful inclusion of the special needs learner in 
ongoing vocational programs. There is a critical need to 
ascertain the best match between the special needs 
learner's interests and skills and the requirements of 
available vocational programs and occupations, (pp. 52- 
59) 

In Norfolk, the method of ascertaining this match and 
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determining appropriate vocational placements along with 

reasonable accommodations was comprehensive vocational 

evaluation. This service was offered for handicapped 

students through Norfolk Public Schools at the Madison Career 

Center (MCC) . The evaluation was a hands-on assessment 

of vocational aptitudes and interests, conducted over 3 to 4 

school days at the alternative site. 

The Vocational Evaluation and Work Adjustment Association 

(1975) has defined vocational assessment as: 

A comprehensive process that utilizes work, real or 
simulated, as the focal point of assessment and 
vocational exploration, the purpose of which is to assist 
individuals in vocational development. Vocational 
evaluation incorporates medical, psychological, social, 
vocational, and economic data in the attainment of the 
goals of the evaluation process, (p. 86) 

This definition was the basis for the development of 
vocational evaluation in rehabilitation facilities during the 
1970s. The definition remained unchanged 18 years later, even 
though the majority of vocational assessments by then were 
conducted in secondary public school settings rather than only 
in rehabilitation facilities. As Stodden (1980) reflected, 
"The present state of the art in vocational evaluation is 
largely manifested within the field of vocational 
rehabilitation, and as a result, the concepts, 
instrumentation, and strategies do not lend themselves readily 
to an educational setting 11 , (p. 6) 

The Norfolk model for vocational evaluation was certainly 
no exception, consisting of a collection of various components 
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of rehabilitation models in existence prior to 1978. The 
original school-based assessment model for Virginia was 
developed in Virginia Beach City Public Schools in 1978 , witn 
only slight modifications as the statewide model was developed 
in 1983 (Scott & Prezioso, 1986) . Because this model seemed 
to have been working effectively (at least with the population 
it was originally designed to serve) , it had remained 
unchanged. 

Through the introductory practicum (Feldt, 1992), 
personnel issues, which may have contributed to students' 
non-attendance and noncompletion of the vocational evaluation 
were examined. No clear-cut reasons for student nonattendance 
or noncompletion could be determined and no marked improvement 
was observed (see Table 1) . Because NPS had a large 
percentage of ethnic minorities, the issue of race of the 
staff in vocational evaluation was addressed. An additional 
evaluator was added as a result of a change in city-wide 
staffing patterns. The new evaluator was black; the 
two evaluators already on staff were white. This change of 
balance had no observable effect on student attendance or 
completion patterns during the 1991-1992 school term. 

Evaluation of the classroom climate and social-emotional 
needs of students was conducted through the introductory 
practicum, without conclusive results. Students who attended 
the center for evaluation and the teachers who worked with 
them did not seem to have concerns in these areas. 
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Table 1 

Rates of Completion in Vocational Evaluation 



Year Percentage ot special education students 

completing vocational evaluation 



1989 85 

1990 67 

1991 71 



Note , This represents completion of a 3- or 4-day 

vocational evaluation by those students who began the 
process . 

The problem as stated had persisted for some time, and 
previous interventions had not resulted in a solution. This 
project, therefore, shifted the focus away from convincing 
students to attend the vocational evaluation center and beyond 
training teachers so that they may better explain the 
services. The new focus in this project was on reviewing and 
revising methods of conducting vocational evaluation including 
methods that would meet the needs of the disadvantaged 
population. Obviously, serving larger numbers of students 
remained in focus. In addition, the project focused on the 
dt ^lopment of alternative means to obtain relevant 
performance data on special education students, which could be 
used to help determine vocational placements where they would 
have the greatest potential to succeed. 

Clearly, the intent of the legislation (Public Law 
101-392, 1990) and of the vocational assessment requirement 
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was to open doors to special-needs students so that they could 
obtain marketable skills while in high school. Further review 
of the legislation indicated that information on vocational 
options must be provided to students (handicapped and 
disadvantaged) no later than the ninth grade. The assessment 
had previously been offered to students upon attaining the age 
of 16 years. Obviously, the timing of the vocational 
assessment had to be changed for compliance with the mandates. 
This project was implemented with the eighth and ninth grades 
as the primary focus, but assessment strategies addressed the 
needs of students in Grades 8 through 12. 

The comprehensive evaluation method, which was the only 
previous vocational assessment option in Norfolk Public 
Schools, was only available to 9th- and lOth-grade special 
education students who were 15 years and 10 months old 
or older and who planned to enter a vocational education 
program. Disadvantaged students were included in the 
city-wide testing of loth graders on the Differential Aptitude 
Test (DAT) , but no interpretation of results was provided and 
no alternative methods were offered to students whose DAT 
results were invalid or inconclusive. In addition, all 
students completed a vocational interest inventory through 
their social studies classes in eighth grade, which included 
disadvantaged and LEP students and some mainstreamed special 
education students. 
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Problem Data 

During the period September 1990 through June 1991, 
which was the period of the introductory practicum, two 
vocational evaluators conducted vocational evaluations on 190 
secondary special education students from the five high 
schools and eligible alternative education sites of NPS. 
During the period September 1991 through June 1992, three 
vocational evaluators conducted vocational evaluations on 172 
secondary special education students from NPS. These compare 
to the 1989-1990 year, before any interventions took place, 
when two evaluators completed evaluations on 180 students. 

The previous interventions placed emphasis on getting 
those students who started the evaluation to complete the 
process. However, the number of students who were evaluated 
represents only about 17% of those handicapped students 
eligible for such services and no disadvantaged or LEP 
students . 

Of the five high schools in Norfolk, one (Granby High 
School) was randomly selected as the site for establishing 
some baseline data on students who refused to attend the 
center-based vocational evaluation and for whom alternative 
methods of assessment were implemented. Fifteen students were 
identified who wanted to enroll in vocational education 
programs for the 1992-1993 school term, but who had refused to 
participate in the vocational evaluation at MCC. The 15 
students 1 confidential files were reviewed, and the following 



commonalities were determined: (a) 100% were classified as 
learning disabled, (b) 100% were mainstreamed 50% or more of 
the time, and (c) 80% were mainstreamed 75% or more of the 
time. 

This became known as the experimental group because new 
methods of assessment were to be instituted. The 15 students 
were interviewed individually. The recommendations are 
summarized in Figure 1. After a conference with the project 
manager in which the center-based vocational evaluation 
process and the new innovative method were explained, two 
students (20%) changed their minds and decided to attend the 
vocational evaluation center for comprehensive evaluations 
(completed in May 1992) . Both received recommendations for 
vocational training program placement for the 1992-1993 term. 
The project manager determined vocational placements based 
upon a review of existing records, a student interview, and 
teacher recommendations for the remainder of the 15 students. 
Of these students, 66.6% were recommended for vocational 
programs at Norfolk Technical Vocational Center (NTVC) , none 
were recommended for MCC, 6.7% were recommended for medical 
release prior to additional vocational assessment, and 6.7% 
were recommended for vocational classes offered within the 
home school for the 1992-1993 school term. 

As a comparison, a group of 15 students with comparable 
handicapping conditions was selected from the 172 who received 
a comprehensive,- center-based vocational evaluation during 
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1991-19.2 and who were recommended for placement in vocational 
programs during 1992-1993. Because the former assessment 
methods were used with these students, they formed a group 
that became known as the control group. The results of their 
vocational evaluations are summarized in Figure 1. 
Comparisons of the recommendations and the end-of-year 
outcomes were made (see Figures 1 and 2) to demonstrate 
whether the type of vocational evaluation had any 
significant relationship to a student's placement into the 
recommended vocational program. Both groups of students were 
monitored during the period of this project to determine the 
outcomes of the vocational recommendations. Students' 
admission to/ enrollment in the recommended programs and grades 
in the vocational programs were compared as indicators of 
success. 

The intent of these comparisons between the experimental 
and control groups was to demonstrate whether such in-depth 
assessment as was offered through the comprehensive vocational 
evaluation was necessary for all handicapped students. The 
untested theory was that students who had learning 
disabilities but were mainstreamed into regular education 
classes could benefit from less intrusive testing methods. 
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Control Group 



Experimental Group 
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Figure 1 . A comparison of vocational assessment 
recommendations between an experimental group and a control 
group . 
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Figure 2 . A comparison of outcomes (student placements) 
between an experimental group and a control group (end of 
1992-1993) . 
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Further interpretation of these comparisons is 
included in chapters 4 and 5. 

Efforts were needed to assure that some method of 
vocational assessment was available for all secondary special 
needs students, including those who could not or would not 
miss 3 to 4 days of instruction from the regular school 
program. According to the legislation (public Law 98-524, 
1984; Public Law 101-392, 1990), all special education 
students, students with limited English proficiency, students 
who are economically disadvantaged, and students who are 
educationally disadvantaged (two or more grade levels behind 
their class or in the lower quartile on- standardized tests) 
are eligible for vocational assessment, which should be a key 
component for planning the individual's vocational education 
program. This project was directed toward the secondary 
special needs students, an estimated population of about 7000 
students in Norfolk. The delivery of vocational assessment 
services for this potential population, including the 
determination of methods, models and procedures, were within 
the locus of control of the project manager as the Program 
Leader for Vocational Educat'on Programs for Special 
Populations with Norfolk Public Schools. 
Probable Cause Data 

Additional data collection was a component of the project 
timetable. A survey of total special education enrollments 
was conducted at each of the five high schools to determine 
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the total population eligible for vocational evaluation during 
the 1991-1992 school year (see Appendix B) . This was compared 
to the number of students who received vocational evaluations 
to determine the percentage actually served. 

Next, secondary special education enrollments for the 
1992-1993 term were determined by the September 30, 1992 
student count. This was considered to be the pool of special 
education students eligible for vocational assessment for that 
school term and served as the basis for project calculations. 

Methods had to be developed to identify from the 
secondary education population those students who met the 
criteria for the disadvantaged and LEP categories. An initial 
review of numbers of students on free or reduced-price lunches 
(Grades 8 through 12) indicated that this criteria alone would 
identify about 6000 students. Additional identification was 
needed to identify those students with low grade-point 
averages, those who were two or more grade levels behind their 
classes, and those who scored in the lowest quartile on 
standardized tests. That was projected to increase the 
target population to about 7000. 

A survey was conducted of NPS high school special 
education personnel, including teachers of students who were 
emotionally disturbed, learning disabled, educably mentally 
retarded, trainably mentally retarded, hearing impaired, 
visually impaired, physically disabled, speech impaired, other 
health impaired, orthopedically impaired, and severely 
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handicapped. They were asked their views on the vocational 
evaluation process and possible reasons why students were not 
referred, did not attend, or did not complete the process, and 
they were asked to react to possible solutions. Information 
was sought on their methods of utilizing evaluation results. 

Student attitudes and perceptions were determined 
through informal interviews of secondary special education 
students conducted by two evaluators and the program leader in 
each high school. Students were asked to give reasons for 
refusing to attend a comprehensive assessment at Madison 
Career Center. Their anecdotal responses clustered in two 
areas: (a) Madison Career Center was viewed as a school for 
retarded students where other students would not attend, and 
(b) special education students in mainstreamed or co-teaching 
classes felt they could not afford to miss any of their 
classes and still get their classwork done so they could not 
attend a 3 or 4 -day testing program. 

Information was needed on the state-of-the-art in 

vocational evaluation in Virginia. According to the Virginia 

Council on Vocational Education (1991): 

Assessment services are available to eighty-five of the 
state's school divisions through thirty-five 
comprehensive vocational assessment centers. The 
establishment of additional centers has been identified 
as a priority need. Thirteen assessment centers were 
added in 1989-90, but fifty school divisions still have 
limited access to such services, (p, 7) 

All school superintendents in Virginia were contacted 

to identify the name/address of their vocational evaluator. 
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If this service was not offered through che district, then 
that was also noted. 

A survey was then conducted of all school districts and 
regional vocational centers offering vocational assessment 
services in an effort to determine any alternative methods of 
conducting vocational evaluations in use in Virginia. The 
questionnaires requested demographic data on each district, 
information on evaluation procedures being used, and 
information on any surplus vocatiQnal assessment equipment 
that was available (see Appendix C) . The responses to the 
mailed questionnaires to these districts and regional 
vocational centers provided few strategies that could be 
implemented in Norfolk Public Schools. When the results of 
this survey were presented at a statewide conference, the 
general reaction from vocational evaluators and educators 
present vas that they were also grappling with the issues 
related to evaluating more students without increased budgets. 
Many school districts have requested information on the 
implementations being tried in NPS through this project. 

Another area that was reviewed for possible improvement 
was the collaboration between the home schools and the 
vocational evaluation center. The services offered at the 
center were coordinated by the vocational resource teachers, 
one of whom was assigned to each high school and two at 
Madison Career Center. They had been previously trained in 
methods of orienting students to the vocational evaluation 
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(Feldt, 1°92) and in utilization of the results of the 
vocational evaluation. Observations were conducted of 
orientation sessions held with secondary special education 
students by vocational resource teachers at each of the five 
NPS high schools because information about the vocational 
evaluation process was disseminated to students at those 
sessions. Observations were conducted with each of the 
vocational resource teachers at the high schools as they 
interpreted and explained the results of completed vocational 
evaluations to parents, students, and special education 
teachers who wrote the students' Individualized Education 
Plans (IEPs) . The results of these observations reflected 
(a) a firm grasp by all of the vocational resource teachers on 
the vocational assessment process, (b) reasons for the 
vocational evaluations, (c) benefits to students, and (d) 
methods for implementing the recommendations. Obviously, 
these employees were already doing what they could to make the 
vocational evaluation process work so no further strategies 
were implemented in this area. 



Chapter 2 
Setting 

Demographic and Organizational Characteristics 

The Norfolk, Virginia school district is the second 
largest in the Hampton Roads area of the state. The district 
serves approximately 36,000 students in kindergarten through 
Grade 12 and operates two early childhood intervention 
centers. Facilities include 36 elementary schools, 8 
middle schools, 5 high schools, 1 technical-vocational 
center, 1 career center for special populations, 1 skills 
center for adults and dropouts, and 8 other auxiliary 
educational programs. 

The Norfolk Technical Vocational Center (NTVC) provides 
a range of occupational education programs to approximately 
800 students each year. Of these, 1% are freshmen (at-risk 
students), 21% are sophomores, 36% are juniors, 35% are 
seniors, and 7% are postgraduates (accepted on a space 
available basis) . At Madison Career Center (MCC) , 120 
disabled students receive instruction in six service-related 
occupational areas. The students at MCC and in two selected 
classes at NTVC are pursuing special education diplomas with 
requirements for completion based upon the Individual 
Education Plan (IEP) . Both vocational sites serve as 
extensions of the five high schools, with students being bused 
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in for three-period classes. 

The student population of Norfolk Public Schools covers 
the socioeconomic range, but between 64% and 82% of students 
at each high school were eligible for free or reduced-price 
school lunches. The ethnic composition of the student body is 
67% black, 32% white, and 1% Asian, Hispanic, and other races. 

The school division is governed by a school board which 
was appointed by the city council. The city council members 
are elected and they elect the mayor from among their ranks, 
but school board members are still appointed. The city of 
Norfolk functions through a council-city manager form of 
government. The school board employs a superintendent, deputy 
superintendent, and five assistant superintendents. Various 
departments manage instruction, adult and vocational 
education, budget, accounting, transportation, purchasing, 
special education, personnel, research and testing, and staff 
development. The school board depends upon the city council 
and the Virginia Department of Education (VDOE) for funding; 
it has no taxation authority of its own. 
Problem Setting Situational Data 

Norfolk Public Schools maintain archival data on the 
enrollments in special education (see Appendix B) and for many 
years was required by VDOE to collect data on the vocational 
education services provided to special populations. This was 
not required due to changes in the funding formula after 1991. 
The information for the period 1989-1992 is summarized in 



Table 2, based upon availability. 
Table 2 

Population Distribution in Special and Vocational Education 
Dates 



Population 



1989-1990 1990-1991 1991-1992 



High school special 
education enrollment 

High school special 
education students 
enrolled in vocational 
education 

Percent of special 
education students in 
vocational education 

High school special 
education students in 
vocational evaluation 

Percent of special 
education students who 
received a vocational 
evaluation 



1,279 



916 



71.6% 



180 



1,249 



1,203 



not 

879 reported 



not 

7 0.4% reported 



190 



172 



14.1% 



15. 2% 



14. 3^ 



Note . The data are from records of Norfolk Public Schools. 
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Obviously, access to vocational education programs by 
handicapped students was not an issue. However, the results 
of the vocational evaluation were intended to be utilized to 
determine appropriate vocational placements, as indicated in 
the enabling legislation. These data show that many more 
youngsters were being placed in programs without benefit of 
vocational evaluation recommendations than were being placed 
with such recommendations. (Note that the data only indicated 
where students were enrolled. No information was available to 
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demonstrate whether these placements were successful. In 
fact, many students were enrolled in vocational classes in 
which they were not passing.) This discrepancy had to be 
addressed. Apparently the former methods of conducting 
vocational evaluation were a limiting factor, so new methods 
had to be explored, studied, and implemented. 

NPS had three full-time vocational evaluators on 
the staff as of September 1991, an increase of one position 
from the previous year. Two of the evaluators were females 
(one black and one white) , and one was male (white) . Two 
evaluators had 4 or more years 1 experience and one was 
completing on-the-job training as an evaluator. The two 
experienced vocational evaluators and the program leader were 
nationally certified in vocational evaluation. One of the 
experienced evaluators resigned in June 1992, and the 
position was not filled, putting staffing back at two 
full-time evaluators. These were the personnel who 
implemented the innovations through this project. 

The program leader (project manager) had 20 years related 
work experience and was one of the first three school-based 
evaluators to be hired in the state of Virginia. She was the 
first evaluator to leave a position with the Virginia 
Department of Reha ilitative Services in order to establish a 
school-based vocational evaluation center. She was considered 
a leader in Virginia in providing services to students with 
disabilities and had been heavily involved in professional 



organizations promoting vocational evaluation, vocational 
resource services, and other adaptations that allow special 
populations to enter regular vocational education. She was 
selected as Virginia's Vocational Special Needs Teacher of the 
Year by her peers in 1986 and as Outstanding Vocational 
Special Educator for Virginia in 1993 • 

Through strong efforts by the project manager and a 
committee of her peers, the Virginia Department of Education 
adopted in 1992 a professional certification for vocational 
evaluators making Virginia one of only a handful in the 
country to license these personnel as they did teachers and 
school psychologists. Lehmann and Hartley (1991) reported 
that, at that time, only Wisconsin, Maryland, and Minnesota 
required vocational certification for employment as an 
evaluator in the public schools; most states had no 
established minimum standards. VDOE also adopted an add-on 
endorsement of vocational special needs educators through the 
state f s teacher certification process. Virginia accepted the 
licensure of evaluators and the endorsement in vocational 
special needs education, illustrating the recognition for 
professional competence in the area of serving special 
populations in vocational education. 

The backgrounds of the two experienced vocational 
evaluators and the program leader were viewed as constraining 
factors in the development of this project. With 
rehabilitation backgrounds, it was difficult for them to 



shift paradigms in order to view new methods as beneficial or 
even acceptable. In addition, the program leader was the 
"grandmother" of school-based vocational evaluation and at the 
onset of this project clung to former beliefs that the methods 
being used were above reproach. She had been little 
swayed by earlier efforts in other states to develop 
alternative evaluation strategies and had, in fact, been 
openly critical of those approaches. 

Norfolk Public Schools had one vocational resource 
teacher assigned to each of the high schools and two 
vocational resource teachers were assigned to the Madison 
Career Center. No vocational resource support was available 
to students attending the Norfolk Technical-Vocational Center 
or the satellite centers. 

Until the 1991-1992 school term, a full-time 
para-professional was assigned to the evaluation center. This 
individual provided assistance for students and clerical 
support in document maintenance and report preparation. The 
position was removed during the 1992-1993 school term, 
returning the clerical operation functions back to the 
full-time evaluators and impacting their ability to work, with 
larger numbers of students. 
Problem Setting Culture 

The administrative structure of vocational education 
programs for special populations was a positive force in 
facilitating implementation of the innovation. The program 
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leader supervised the vocational evaluators and the vocational 

resource teachers. She had a positive working relationship 

with special educators at the secondary level and served as a 

liaison with their administrative staff. The Director of 

Adult and Vocational Education was committed to supporting the 

program leader's efforts at improving evaluation services for 

special needs students. 

Staff were additionally motivated by the knowledge the 

federal program monitoring would occur in May 1993, with 

compliance to the vocational legislation being examined. 

According to a report from the Virginia Council on Vocational 

Education (1991) : 

Each school division must provide assurances on issues of 
equal access in recruitment, enrollment, and placement in 
the full range of vocational programs; the delivery of 
services in the least restrictive environment; and 
coordination with special education for those identified 
as handicapped students. Compliance with these and other 
related assurances are assessed through several 
evaluation and compliance review processes, (p. 3) 

A constraining factor in the maintenance of the 

center-based assessment was the students 1 perceived stigma 

attached to being singled out as "handicapped" and requiring 

special services, especially if that meant attending Madison 

Career Center. Those students who were mainstreamed or in 

co-teaching classes were particularly sensitive to this. It 

was not feasible to move the assessment center to a more 

neutral site. 

Another possible constraining factor in trying to assess 
more students was the size of the evaluation center. The 
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center was moved from Rosemont Middle School to MCC in 1989. 
The MCC site had considerably less floor space than the 
original facility and was crowded when more than seven 
students were evaluated concurrently. This contributed to the 
decision to implement assessment techniques with students at 
the home school rather than maintaining the notion that all 
assessment should be center-based. It was simply not 
physically possible to work with more than seven students at 
any given time. 

Another possible constraining factor related to the 
facility was its age and inaccessibility. The 60-year-old 
building predated architectural adaptations for special 
populations. Students with certain physical limitations could 
not access the vocational evaluation facility; no elevator was 
installed, and the assessment center was located on the third 
floor. This required that evaluation services be provided to 
certain students at their regular high schools, sometimes 
still as a pull-out option, and that testing materials and 
equipment had to be transported from Madison to the school 
sites. 

An external force that limited implementation was the 
traditional vocational evaluation method, which was widely 
accepted in rehabilitation and school settings. Testing 
materials and work samples available may have become outdated 
or irrelevant for the populations to be served, for the 
requirements of the future work force, and for the educational 
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system in which they are being used. However, the cost of 
developing or purchasing new materials or work samples was 
prohibitive, in view of budget and staff time restraints. 
Rather than investing efforts in further work sample 
development, alternative assessment approaches had to be 
explored. 

The program leader was under pressure from upper 
administration to develop or locate a quicker, pencil-and- 
paper vocational assessment that could also be available to 
disadvantaged and "regular" education students. This method 
of assessment was in vogue during the 1960s and 1970s, but 
was replaced by "a more realistic, work sample approach" 
as vocational evaluation services were developed (Uthe, 1980, 
p. 36) . She gave these examples of pencil-and-paper 
assessments: (a) General Aptitude Tv?.st Battery (GATB) , 
(b) Kuder Preference Record, (c) Differential Aptitude Test 
(DAT) , (d) Strong Vocational Interest Blank, (e) Hester 
Evaluation System, and (f) Ohio Vocational Interest Survey (p. 
36) . 

The GATB had been discontinued from use by the Virginia 
Employment Commission following heavy controversy about racial 
bias. It was still used as a key component of vocational 
evaluations by area rehabilitation facilities, but had not 
been utilized by NPS. The DAT was found to be administered 
annually to all high school sophomores in Norfolk and 
presented an opportunity for expansion if permission could be 
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solicited to move the administration down to the eighth grade. 

One area school district suggested the possibility of 
administering the Armed Services Vocational Aptitude Battery 
(ASVAB) to all high school seniors as a career counseling 
tool. This was traditionally administered only to those 
seniors who expressed an interest in entering military 
careers. This method was examined for potential applications 
as part of this project. The project manager learned that 
cutbacks in the U.S. Department of Defense (DOD) would make 
expanded use of the ASVAB test impossible. DOD maintained 
strict control of test materials and refused to share them. 
In addition, the ASVAB was normed on 18 to 20-year-old 
recruits, and would not have been of benefit with younger 
students. 



Chapter 3 
Review of the Literature 



The review of literature served the project manager in 
two ways. First, the historical perspective validated the 
manager's belief that the methods being used to conduct 
vocational evaluation in Virginia were not progressive. The 
second benefit was the collection of examples of best 
practices, which were in place in other states, allowing the 
project manager to accept or reject methods for inclusion in 
the newly evolving model for NPS. 
Historical Perspective 

According to McCray (1982) , vocational evaluation began 
in the 19 30s as an outgrowth of the State-Federal Vocational 
Rehabilitation Program and as a response to the inadequacy of 
traditional assessment and guidance tools for handicapped 
populations. This same dynamic was involved in introducing 
vocational evaluation into school settings during the 1970s. 

Nadolsky (1977) asserted that early assessment was based 
upon the notion of industrial scientists that each person was 
best suited to perform a limited number of industrial 
functions. He added, "the vocational assessment movement, as 
it emerged within our manufacturing society, was designed to 
meet the demands of industry, moreso than the needs of 
individuals" (p. 7) . By relying on the results of vocational 



assessment, industry could select those individuals who 
possessed an abundance of talent and who, therefore, had a 
greater probability of being able to perform industrial jobs. 
The implication was that persons who did not perform well on 
the vocational assessment were denied employment and 
considered not employable. Naturally, this resulted in the 
majority of disabled individuals being denied employment 
because they demonstrated a low probability of being capable 
of competitive industrial employment. "With the 
encouragement ... of vocational rehabilitation, vocational 
evaluation [eventually] emerged as an individually-oriented 
trend" (Nadolsky, 1977, p. 8). 

McCray (1982) traced the history of vocational evaluation 
through the Medical Facilities Survey and Construction Act of 
1954, legislation that authorized the construction of 
comprehensive rehabilitation facilities, which included 
vocational evaluation services. He noted that the first 
authorized funding for training evaluators and for conducting 
related research was through the Rehabilitation Amendments of 
1954 and the Rehabilitation Act of 1973. McCray attributes 
stimulated growth of the field to passage of the Education of 
All Handicapped Children Act, Public Law 94-142, and the 
Vocational Education Amendments of 1976, Public Law 94-482. 
It was from that growth spurt that he observed vocational 
evaluation programs beginning in school-based settings. 

The shift of site for vocational evaluation from 
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rehabilitation-based to school-based settings also required 
a shift in emphasis (McCray, 1982) . Whereas establishment of 
eligibility and facilitation of vocational planning were the 
focus of evaluation in the rehabilitation facilities, the 
purpose in the school settings became developmental, with an 
emphasis upon planning educational outcomes for individual 
students. The school-based utilization of the results of the 
vocational evaluation also accented the possibilities of 
career and vocational exploration as part of the individuals 
developmental educational process. 

Shortly after this evolution into school-based sites, the 
literature reflected heated controversy over definition and 
purpose. Was the service needed for special-needs students a 
vocational evaluation or was it vocational assessment? 

As defined by the Vocational Evaluation and Work 
Adjustment Association (VEWAA) Glossary (1988) , vocational 
evaluation "is a comprehensive process that systematically 
uses work, either real or simulated, as the focal point for 
assessment and vocational exploration, the purpose of which is 
to assist individuals in vocational development", (p. 3) This 
process was described as incorporating medical, psychological, 
social, vocational, educational, cultural, and economic data 
into the process to attain the goals of evaluation. Also 
defined in the VEWAA Glossary (1988) was vocational 
assessment: 

the comprehensive process conducted over a period of 
time, usually involving a multi-disciplinary team . . . 
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with the purpose of identifying individual 
characteristics, education, training and placement needs, 
serving as the basis for planning an individual's 
educational program and which provides the individual 
with insight into vocational potential, (p. 3) 

Nolte (1989) asserted that the scope of vocational 

assessments in schools varied from state to state, with lack 

of standardization due to nonspecific federal requirements. 

She noted that "The scope of vocational assessments ranges 

from vocational screening all the way to a full comprehensive 

vocational evaluation. In the school-based environment, the 

difference between vocational evaluation and vocational 

assessment is of importance" (p. 109) . Her rationale was that 

the purpose of vocational evaluation was to guide individual 

vocational development, but that the purpose of vocational 

assessment was to guide the educational program for an 

individual. In other words, the intent was similar but 

the outcome was different because vocational assessment would 

determine the most appropriate vocational training program 

within the parameters of the available education services. 

One of the techniques of vocational assessment, following this 

argument, was comprehensive vocational evaluation, but there 

were others . 

Cobb and Larkin (1985) proposed "to eliminate the term 
vocational evaluation as it pertains to the entire range of 
assessment activities associated with screening, placement, 
and program planning and monitoring for an individual child," 
maintaining that we should "assess individuals and evaluate 
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programs" (p. 3). They maintained that the rehabilitation 
assessment model did not adapt well to the educational 
environment because it attempted to make predictions about 
employment suitability, but not about curricular options. 

Emery (1984) asserted that traditionally the ultimate 
goal of the rehabilitation model was client placement into 
employment. She stated, however, that the emphasis for 
providing assessment services in the school setting had 
shifted to career development and placement in vocational 
programs. In other words, task-related abilities were 
diminished by the need to identify what the student could 
learn while in the educational setting (p. 75) . 

Cobb and Larkin (1985) offered support for an additional 
definition of what was described as "contemporary assessment" 
(p. 3) . That "refers to those practices that clearly link 
the purposes and outcomes of assessment with the goals and 
techniques of instruction and other forms of service 
intervention" (Halpern, Lehman, Irvin, and Heiry, 1982, p. 1) . 
Halpern et al. further elaborated upon the differences between 
traditional vocational assessment and contemporary assessment: 
"Rather than rely on traits or aptitudes to infer performance, 
the contemporary approach emphasizes the importance of direct 
assessment of actual competencies, [and] requires the outcomes 
of measurement to have direct implications for program 
planning. " (p. 4) 

This perspective would make vocational assessment an 
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ongoing activity conducted by a team of school personnel. 
This theory formed the basis for the split between groups in 
the field who favor comprehensive vocational evaluation as a 
one-time event or those who favor vocational assessment as a 
process conducted over time. Indeed, this redefinition became 
the basis for the paradigm shift needed to restructure 
vocational assessment activities in NPS through this project. 

A review of works from the past 15 years revealed that 
vocational evaluation as a requirement of the Carl D. Perkins 1 
Act was an asset for special-needs students. According to 
Feldt (1987) , "this essential component of the habilitation 
process was traditionally made available when students were 16 
years old and still in school", (pp. 160-163) 

Until the passage of the Perkins' Act (Public Law 101- 
392, 1990), vocational evaluation was not required for 
students and in many localities was only available to adults. 
No specific references to the problem of getting special-needs 
students to attend and complete the vocational assessment 
process or which students should be referred for vocational 
evaluation were located, but inferences were drawn from 
articles about other school-based vocational evaluation and 
special education innovations, which may apply to the concerns 
at hand. In addition, the law is very general as to what 
constitutes a vocational assessment of special-needs students, 
leaving definition of methods and procedures to the states and 
localities. Very little writing could be located that 
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addressed the vocational evaluation of disadvantaged or LEP 
students . 

According to Kochhar and Barnes (1992) , the Individuals 
with Disabilities Education Act (IDEA) (Public Law 101-476, 
1990) and the Americans with Disabilities Act (ADA) (Public 
Law 101-336, 1990) fit well with the Perkins 1 Vocational and . 
Applied Technology Act (Public Law 101-392, 1990) to "develop 
broader, far-reaching mandates to include youth with special 
needs in the range of career/vocational and transition 
services" (p. 18) . This excerpt is from their: 

Bill of Rights 2000 for Youth with Special Needs: 

1. The right to accommodation of special learners 

in the full range of mainstream and special education 
programs and services, including regular education, 
vocational education, transition services, job training, 
placement opportunities, and articulated (e.g. Tech-Prep) 
and postsecondary placement assistance. 

2. The right to receive a comprehensive vocational 
assessment as part of transition services required under 
IDEA. (p. 19) 

Washburn (1979) wrote that determining vocational 
placements for special-needs students required specially 
trained staff, findings from actual work samples, and 
comprehensive assessment of skills. She proposed that the 
best method of obtaining such an assessment in a larger school 
district was "at a central testing facility, which could be of 
service to all students" (pp. 14-18). This may have 
implications for Norfolk where only students classified as 
special education are evaluated at the Madison Career Center. 

The literature review revealed several other pervading 
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controversies. Numerous authors discussed the best methods 
for conducting vocational assessment or vocational evaluation, 
predominantly of handicapped students. When the assessment 
should be conducted was of great concern across the field of 
vocational assessment. Also, controversy prevailed on the 
length of time that a vocational evaluation or assessment 
should take. The various methods will be dealt with at length 
in this project and will constitute the bulk of 
experimentation and innovation. The timing of the vocational 
assessment will be assumed to be during the eighth or ninth 
grade, as indicated in the Perkins 1 regulations. The length 
of time required for vocational evaluation will be based upon 
the established standards for vocational evaluation from the 
Task Force Proposal for CARF Standards on Vocational 
Evaluation (Vocational Evaluation and Work Adjustment 
Association, 1975) , which state: "The length of time an 
individual remains in vocational evaluation shall be primarily 
based upon the time necessary to accomplish the individual's 
evaluation goals" (p. 73) . 

One component of the process of redesigning the 
vocational assessment model for NPS was determining which 
students would require which form of vocational assessment or 
evaluation, if several methods were available. In describing 
the history of vocational assessment as an evolution away from 
psychological testing and toward miniatures of work stations, 
Nodalsky (1984) relayed the three levels of the assessment 



process as described by Task Force Number One of the 
Vocational Evaluation Project. These were: first level — 
screening, second level — clinical, and third level — vocational 
evaluation. The first two levels, he insisted, made use of 
psychological testing and counseling processes and were 
beneficial to individuals who could abstract and were 
verbally oriented. The latter was for individuals who were 
nonver- lly and experientially oriented; in other words, 
vocational evaluation as "hands on" activities, primarily 
benefited individuals "whose thought process is primarily 
governed by the right cerebral hemisphere" (p. 7) Nadolsky 
made an additional observation of note for this project. "The 
majority of individuals who receive vocational assessment 
services ... do not require vocational evaluation; they 
have sufficient verbal and logical reasoning ability to 
benefit from the application of traditional verbally oriented 
procedures employed during the first and second levels of the 
vocational assessment process" (p. 7). 

This view helped free the project manager of the belief 
that all special-needs students had to pursue work sample 
performance testing. It renewed the belief that some students 
will perform satisfactorily on pencil-and-paper testing or on 
previous testing and performance measures so that assessment 
could be successfully completed without requiring attendance 
at a vocational evaluation center. 

Scheer (1990) described two phases of assessment, which 
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he called prevocational evaluation and vocational evaluation. 
The early phase was a feasibility stage where vocational 
diagnosis had to be made. This included psychometric testing 
related to dexterities, achievement, interests, and aptitudes 
and utilization of information from the assessments of other 
professionals (physical, speech, auditory, and psychological) . 
He suggested that the second phase was for determination of an 
individual's potential to perform specific types cf work and 
whether they were employable. This second phase could include 
psychometrics but "stressed performance on situational 
assessments, job-site evaluations, and use of standardized 
work samples" (pp. 40-42) . 

The project manager related these two phases to the needs 
of disadvantaged and handicapped NPS students. This formed 
the basis for planning that evolved into the multiphase 
model of vocational assessment. In the first phase of what 
was to become the NPS model, students completed psychometric 
testing on vocational aptitudes and interests. If the results 
were inconclusive or invalid, then students could be assessed 
based upon an interview and a review of previous performance 
on other types of assessments. If that combination did not 
provide the basis for vocational programming predictions, then 
comprehensive vocational evaluation became an option. 

It should be noted that the bulk of literature on 
vocational assessment and vocational evaluation related to the 
provision of those services to disabled populations. Nothing 
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of note was available on the vocational assessment of other 
special populations. Wircenski and Wircenski (1991) described 
a 2-year collaborative effort wherein the Garland, Texas 
public schools focused upon the needs of disadvantaged and 
at-risk students. Although the model, which they developed, 
included assessment and data collection in general terms, they 
did not specifically mention the incorporation of vocational 
assessment or evaluation. 
Assessment Methods 

Many of the articles from journals were descriptions of 
similar models, so a compression was made and general 
descriptions provided. However, a number of interesting, if 
not related, approaches were also summarized. All of the 
models described in this literature review were considered in 
developing the multiphase model for NPS. A number of 
components were adopted as described and others were adapted 
to meet local needs. 

Peterson (1984) summarized that models for vocational 
evaluation and assessment of special-needs students in school 
settings were still in great flux. He had conducted a review 
of literature and concluded that efforts were related to two 
basic approaches: (a) curriculum-based vocational assessment, 
and (b) vocational evaluation centers. 

Botterbusch (1989) also described two different 
approaches to vocational evaluation. He described the two 
major models as: (a) psychometric, and (b) clinical, 
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concurring with the writings of Cobb (1972) . He described the 
psychometric model as being rooted in military and personnel 
psychology, requiring the careful use of standardized 
aptitude, achievement, and temperament tests. He believed 
that this model would become more commonly used as efforts to 
evaluate more clients with diminishing resources continued. 
Botterbusch 1 s clinical model emphasized the intuitive skills 
of an evaluator observing students in real or simulated work 
situations. That method "provided information on client 
behaviors, knowledge, and interactions, M which was useful in 
developing vocational and placement recommendations (p. 118) . 

Leconte and Boyer-Stephens (1992) favored a model that 
appraised a list of attributes and factors including: (a) 
level of career development, (b) knowledge of vocational 
education opportunities, (c) vocational preferences and 
interests, (d) individual special needs (learning preferences, 
assistive technology needs, academic supports, vocational 
supports, and functional supports) , (e) identification of a 
prime ry personal advocate, (f) status of awareness and 
linkages with adult services, and (g) ability to use networks 
and access services, (pp. 57-58) 

A concept closely related to vocational assessment is 
self -evaluation . As described by West (1987) , this is a 
component of the assessment process that could be tied into 
regular classroom instruction and used as a counseling tool in 
describing the need for comprehensive services. This concept 



had been developed by the project manager and was included as 
a component of Careers and You , a middle school career 
exploratory curriculum developed and copyrighted by Norfolk 
Public Schools in 1990. As students enrolled in the course 
explored each of 15 career clusters, they were asked to 
rate their performance and interest in the related activities. 
This information was maintained on individual student folders, 
which became a part of the student's cumulative school 
record. Vocational interest testing, self-reflection, and 
physical capacities ratings were a part of this self 
evaluation format. This self -assessment component was 
maintained as a part of the new multiphase assessment model, 
with minor revisions. No efforts were made to develop a high 
school self-assessment because no vocational courses could be 
identified in those grades through which all students could be 
assessed. 

According to McCray (1982) and as reported by the 
National Clearinghouse on Postsecondary Education for 
Individuals with Handicaps (1990) , situational assessment has 
long been held to be a valid component of the comprehensive 
vocational assessment process. This was considered a suitable 
alternative method to vocationally evaluate students such as 
those described by Wood (1984) who suffered from test anxiety, 
embarrassment, or difficulty with time constraints and 
therefore did not attempt a vocational evaluation. 
Situational assessment was included in the NPS multiphase 
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assessment model as well. The vocational evaluators traveled 
to school sites throughout the city to conduct situational 
assessments of students working in familiar settings. 

Data collection forms were developed to obtain 
information from job coaches, classroom teachers, and 
in-building vocational education teachers who had worked with 
the student. This process, also known as curriculum-based 
vocational assessment (Stodden, Ianacone, Boone, and Bisconer, 
1987; Albright and Cobb, 1988(b)) had been attempted in 
Norfolk at the middle-school level, with mixed results, and 
was discontinued. Many complaints had come from the 
vocational education teachers who had been asked to observe 
students at work on various projects in the classroom and to 
indicate on a form whether students exhibited certain 
strengths or limitations in the worker traits. The teachers 
complained that it took too much time, that it was difficult 
to watch students closely enough (in a class of 20 or more) , 
and that they felt uncomfortable making recommendations that 
would affect students in future years. No curriculum-based 
vocational assessment (CBVA) model had been attempted in 
Norfolk at the high school level prior to this project. As a 
comporent of the multiphase assessment model, CBVA requests 
for observation and reports are made of teachers for only one 
or two students at a time. In addition, the teachers are 
asked to report facts and observations only. All 
interpretations of data and recommendations for future 
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programming are made by the vocational evaluators. 

The National Institute of Handicapped Research (1984) 
described six models for delivering vocational assessment 
services: " (1) assessment in the special education classroom, 
(2) assessment in occupational exploration classes, (3) 
integrated vocational assessment, (4 ) vocational evaluation 
center, (5) contracted vocational assessment, and (6) mobile 
vocational evaluation units" (p. 3). The first three were 
easily integrated into classroom activities, but lacked a 
comprehensive approach. School-based centers and contracted 
services reportedly had the tools necessary for a thorough 
evaluation, resulting in a more comprehensive, individualized, 
and work-oriented process with optimum outcomes. Mobile units 
were described as a useful compromise. NPS has integrated the 
first five models into the new multiphase model developed and 
implemented through this project. 

Hastings (1984) proposed a new direction for vocational 
assessment based upon self-evaluation in the work setting. He 
proposed utilizing video cameras to record clients in real or 
simulated work settings and then reviewing the videotapes in 
discussion sessions to evaluate with students the 
appropriateness of their behaviors within the work climate. 
The project manager had used this technique successfully in a 
work adjustment program where behavior development was 
critical. However, this concept was reviewed and rejected for 
the NPS assessment model because it was impractical for the 



few students in the community-based training placements and 
unnecessary for those students in classroom-based training in 
light of the inclusion of curriculum-based vocational 
assessment techniques . 

Evaluation of prevocational skills was espoused by Phelps 
(1984), who proposed collecting data on a battery of traits, 
qualities, and attitudes desired of employees such as 
punctuality, respect for supervision, quality of work, and 
neatness. Students who attend the NPS vocational evaluation 
center housed at Madison are rated using the Materials 
Development Center (MDC) Behavior Checklist, which covers 
those areas described by Phelps. The MDC Behavior Checklist 
was also incorporated into the Phase I vocational assessment 
component of the new NPS multiphase assessment model. 

A Triennial Integration Model was discussed by Levinson 
(1989) as an organized and effective method for delivery of 
vocational assessment. He proposed that the triennial special 
education assessment should have a vocational component, 
beginning when students are in the middle school and 
continuing with each triennium until graduation. This theory 
also fit nicely into special education legislation, which 
required transition planning. In terms of assuring that all 
special education students received a vocational assessment, 
this method certainly had merit. The weak point of this 
proposal, however, was that Levinson advocated that the school 
psychologists collect the data rather than utilizing 
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vocational evaluators, a compromise which the project manager 
was unwilling to consider. This method was not explored as a 
component of the project because it meant adding to the 
workload of the few school psychologists on staff, but may 
have merit for future exploration. 

Rubinsky (1991) proposed using only one system, a 
commercially developed one called the McCarron-Dial Work 
Evaluation System, to vocationally assess mentally retarded 
students. The logic behind this proposal seemed weak, based 
more upon commercial than client service ethics. Some 
components of the McCarron-Dial system are utilized in Phase 
I and Phase II of the NPS multiphase assessment model, but 
it is not appropriate for disadvantaged or less severely 
handicapped students. 

Lehmann and Hartley (1991) proposed vocationally 
assessing students through cooperative education programs and 
the cooperative job placements in the community. They called 
this an experiential model, because the student was developing 
vocational skills by working inside the school or outside in 
the community. This suggestion was integrated into the NPS 
model for those students working in enclave and supported 
employment placements. It required a low evaluator-student 
ratio and so necessarily will be utilized on a limited basis. 

Mason (1984) presented a hierarchical model of vocational 
evaluation, designed to serve the needs of economically and 
academically disadvantaged students as well as students with 
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disabilities. However, the model was difficult to decipher, 
complex, and confusing. It was basically a five-phase model 
with assessment options ranging from a few hours to 6 weeks 
for an individual student, based upon individual needs and 
long-term goals. She described the system as flexible, but 
unwieldy seemed more apt. 

The Illinois Model (Sprengel and Moradian, 1989) involved 
using a specific assessment tool, the Illinois Vocational 
Interest Survey and Assessment (IVISA) to determine interests, 
abilities, work-related behaviors, work skills, and present 
and future employment options for students with severe and 
profound disabilities. This model was developed, admittedly, 
to make compliance with the Perkins 1 regulations. 

A similar type of assessment, the Interest, Learning 
Styles and Aptitude (ILA) Vocational Assessment, was developed 
in-house and copyrighted by the Prince George's County 
(Maryland) Public Schools (1988) and offered to all secondary 
students, not just special populations. After reviewing these 
locally developed, pencil-and-paper assessments, it became 
obvious that replication of the process would be expensive and 
time-consuming, requiring that all student assessment halt for 
a period of 6 months to one year to allow for test 
development. Even after that, questions of test reliability 
and validity would have to be settled. Instead, the project 
manager reviewed commercially developed, "machine-scorable" 
instruments of a similar nature. It was decided that the 
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Differential Aptitude T^st (DAT) would be the most beneficial 
and cost-effective option. That was adopted as part of the 
NPS multiphase assessment model and offered to all students, 
including special populations in the 8th and 10th grades. 

Peterson (1981) reported on the development of a model of 
vocational assessment for use in the public schools in Texas. 
Highlights of that model were that vocational assessment could 
be periodically repeated, that vocational assessment was 
interactive with instruction, that vocational assessment began 
in the seventh grade, and that existing school resources were 
intensively utilized. No more than four to six students per 
evaluator could be assessed at a time using this model. 

Peterson, Brown, and Leconte (1987) presented a 
comprehensive, curriculum-centered approach to vocational 
assessment for vocational education. Their proposed model 
included three phases \ ith five components. It presented 
vocational assessment as a continuing process rather than an 
isolated event. In their Phase I, students received 
vocational assessment prior to vocational education which 
included curriculum-based career assessment, specific, short- 
term assessment related to vocational education, and formal 
vocational evaluation. Phase II consisted of vocational 
assessment during vocational education. This appeared to be 
related to the attainment of specific course competencies and 
work behaviors. Their proposed Phase III was for additional 
vocational assessment upon completion of vocational education. 
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This all-encompassing scheme had the vocational evaluator 
responsible for all monitoring of progress for special 
populations in all the vocational programs and for providing 
reasonable accommodations if difficulties arose. This was a 
surprising proposal , based more upon theoretical concepts than 
on practical applications, in contrast to what these authors 
had espoused before, and seemed to the project manager to have 
been written as a challenge to others in the field to develop 
meaningful dialogue. In a school system with more than a 
handful of special-needs students this approach is not 
workable. 

The Minnesota Career Assessment model (Murray & Skaja, 
1984) was introduced in the Intermediate School District. 
Using a traditional center-based, rehabilitation model of 
vocational evaluation, the staff evaluated 7 2 handicapped 
students during the first year of operation. The description 
of the Minnesota model seemed a carbon copy of the former NPS 
model, as established in 1980. 

The Practical Arts Evaluation System (PAES) model 
(Swisher, 1989; Swisher & Clark, 1991) was described as a 
middle school/ junior high school level exploration and 
assessment program, implemented in Shawnee Mission, Kansas. 
Through the integration of ongoing assessment in the form of 
modules into practical arts classes, students were able to 
complete assessment activities as part of the class 
curriculum. This parallels the Careers and You course offered 
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at the middle school level in NPS with one exception. The 
course itself is a career exploration based upon use of 
modules in each of 15 career clusters. Rather than 
assessing students based upon norm-referenced or criterion- 
referenced methods, as is done in PAES, students in NPS 
complete a self-assessment. The utilization of the results as 
a counseling tool for selection of ninth-grade courses is the 
same in PAES and in NPS. 

The Utica, New York approach to vocational assessment 
(Coffey, Szymanski, & Strong, 1984) included younger children, 
between ages 9 and 11, with developmental disabilities, in a 
multidisciplinary prevocational assessment. As proposed, 
this assessment model required a commitment of about 2 weeks 
per child in elementary school, a comprehensive vocational 
evaluation in the adolescent years, and ongoing team 
monitoring of vocational efforts throughout high school. This 
model seemed to have confused the special education mandate 
for transition services with the vocational education mandate 
for vocational assessment and was trying to accomplish both 
tasks through the overextension of the vocational evaluation 
staff. Again, in a school division with 3,600 disabled 
students in kindergarten through Grade 12 and only two 
evaluators, this is unmanageable. 

Neubert and Leconte (1990) described a vocational 
assessment and intervention model being used in Maryland. 
That model utilized Vocational Service Support Teams (VSSTs) 
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to collect data for the vocational student profile, to modify 
curriculum for individual students, to mentor special-needs 
students in vocational education, and to tutor or co-teach in 
the vocational classes to assist students • From the 
description provided and from telephone and personal contacts 
between the project manager and the authors, this model 
presented some intriguing points for further study. For 
example, the job descriptions of the VSST members paralleled 
those of the vocational resource teachers in Norfolk • In 
addition, the vocational resource teachers have been an 
integral component of the collaboration and coordination for 
vocational evaluation since that program's inception. 
Finally, these teachers have been viewed as expendable in 
light of budget constraints. By utilizing them more closely 
in the assessment and monitoring process, these positions may 
have renewed purpose. Such changes were not within the scope 
of this project, but will be pursued by the project manager. 

Other statewide models were reviewed. They are 
discussed in the comparison of working models and best 
practices in chapters 4 and 5. 

In summary, the review of literature revealed that some 
new methods had been implemented in the vocational evaluation 
process in other states. It was noted that vocational 
assessment was the term used in the literature to cover all of 
the various methods being tried. Vocational evaluation was 
used to refer to that specific process which Virginia had 



modeled, where work sampling and psychometric testing provided 
the basis for vocational recommendations* The innovations 
centered around offering various options for students and 
using as much existing information about each student's 
abilities as possible. This differed from the Virginia model 
in which each student was tested for academic achievement, 
vocational aptitudes, and specific occupational potential over 
a period of 3 or 4 days, in an evaluation center. 
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Chapter 4 
Methods of Discrepancy Reduction 
and of Educational Change 

The major activities of this research project were 
planned to produce these final results: 

Terminal Objectives 

1. To develop alternative methods for conducting 
vocational assessment of secondary special-needs students, 
including those eligible for special education services, those 
who are economically and academically disadvantaged, and those 
with limited English proficiency; to explain the new methods 
to school division personnel; to implement the alternative 
methods; and to evaluate the methods for effectiveness and 
efficiency. 

2. To provide vocational evaluation services to 

at least 20% of eligible special-needs students. Of the 1200 
eligible special education students, 172 were assessed by 
three evaluators in the 1991-1992 year, usinq the former 
model. The goal was to evaluate at least 24 0 special 
education students utilizing two vocational evaluators and 
to evaluate at least 1200 (approximately 20%) of the more than 
6,000 identified disadvantaged students. 

Achievement of these terminal objectives required the 
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incremental achievement of major activities or process 
objectives. A detailed time line was used to organize the 
implementation of these major activities and is shown in 
Appendix A. 

Process Objectives 

1. Critical competitors to comprehensive vocational 
evaluation were identified and reviewed to determine 
appropriateness for inclusion in the Norfolk Public Schools 
program (see Table 4 for comparisons) . 

2. A survey of secondary special education teachers 
was conducted to determine their perceptions of the vocational 
assessment program and the innovations. (See Appendixes K and 
L for the survey and results.) 

3. An experimental group of students from Granby High 
School, who did not go through the comprehensive vocational 
valuation process, but who had a curriculum-based assessment 
or a Phase I assessment, were monitored to determine whether 
they were admitted to the recommended vocational training 
programs, and their grades and attendance in vocational 
courses were monitored as indicators of success in vocational 
programs. These results were then compared to the same 
information on a control group of similar students who 
completed a comprehensive vocational evaluation prior to 
entering vocational training programs (see Figures 1 and 2). 

4. A survey of school-based vocational evaluation 
centers in Virginia was conducted as one step in the 
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identification of alternative evaluation strategies (see 
Appendix C for the survey and Appendix D for results) . 

5. Vocational evaluation personnel reviewed 
alternatives to the former method of obtaining vocational 
assessment data. Through utilization of materials from 
literature review, site visitations and surveys, they 
redesigned the Norfolk vocational evaluation model, updating 
it from the existing 4-day pull-out model, which had been 
initiated in 1980. 

6. Alternative approaches to vocational evaluation were 
implemented; the comprehensive vocational evaluation component 
was maintained, but fewer students were found to need such in- 
depth assessment. 

7. Observations were conducted of the vocational 
resource teachers as they explained the vocational evaluation 
process and as they interpreted the findings to students, 
parents, and educators. 

8. The results of the various vocational assessment 
processes and methods were reviewed, revised as needed, and 
recommendations for changes were made. 

9. Staff development activities were conducted to 
inform vocational resource teachers of the new vocational 
assessment procedures and to disseminate copies of the newly 
developed model. 

10. Staff development activities were conducted so 
the vocational evaluators could thoroughly implement all new 
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procedures . 

11. Staff development activities were conducted with 
administrators, vocational educators, special educators, 
guidance counselors, school psychologists, and school-based 
rehabilitation counselors to assure an understanding of new 
vocational assessment procedures. Role definition related to 
the new process was stressed, 

12. A summary of activities, implementation efforts, and 
findings were developed into a paper, presentation, and 
handouts and were presented to vocational evaluators at the 
Virginia conference, May 1993. Composite results of the 
statewide evaluation center survey were provided to all sites 
responding. The project manager will present at the Virginia 
Vocational Special Needs state conference, August 1993, and at 
the International Conference of the Council for Exceptional 
Children, October 1993. 

13. A statewide clearinghouse of surplus vocational 
evaluation equipment was developed. 

14. Project activities were conducted, including the 
collection and analysis of data, regular meetings with school 
division project committee, and development of progress 
reports at 6 months intervals. 

As indicated in Table 3 (Flowchart of Major Activities) 
these final results were obtained by progressing through a 
series of activities. 

The time line was modified and followed as the project 
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developed (see Appendix A) , serving as an organizing 
technique for the practician manager. It was also used as 
a counseling tool and control measure by the members of the 
site-based project committee to assure that the project 
progressed according to plan. 
Limitations 

This is a descriptive report of activities conducted 
in one school system. There is no attempt to claim that 
results are generalizable , although the model presented could 
be the basis for improvements in other school districts. 
There was no random selection or other "experimental" 
conditions. The researcher was an active participant in the 
process. * 
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Table 3 

A Flowchart of Maior Activities Involved in Assessing, 



Revising, Implementing and Evaluating the Vocational 
Assessment Process in Norfolk Public Schools 



Assessment Planning and Implementation Evaluation 

Phase Revision Phase Phase Phase 



Assess local 


Identify a 15 


Cont inue "Old" 


Collect/ 


concerns 


student control 


method of 


compare 




group 


evaluation 


data 


Conduct 


Develop new 


Implement "New" 


Recommend 


faculty 


activities and 


methods and 


"Best" 


survey 


strategies 


strategies 


practices 








to NPS 


Survey 


Conduct staff 


Follow-up on 


Write up 


students 


development 


Granby HS 


and 






Group . 


present 








results 


Survey 




Follow-up on 




division 




control group 


Conclude 


superintendents 




the 








project 


Survey 








evaluation 








activities at 








centers 








statewide 








Review 


Compare the 


Implement best 




literature on 


various 


practices from 




alternative 


models 


literature 





models of 
evaluation 

Conduct project 
activities 



Spin-off activities : 

1. Disseminate results statewide of survey of Virginia's 
vocational evaluation centers. 

2. Develop statewide clearinghouse of surplus work samples. 

3. Disseminate results through conferences. 




Chapter 5 
Results 

Terminal Objective 1 Restated and Expanded 

1. To develop alternative methods for conducting 
vocational assessment of secondary special-needs students, 
including those eligible for special education services, 
those who are economically and academically disadvantaged, 
and those with limited English proficiency; to explain the 
new methods to school division personnel; to implement the 
alternative methods; and to evaluate the methods for 
effectiveness and efficiency. 
Terminal Objective 1 Accomplishments 

This terminal objective had to be modified from the 
original project proposal prior to the development and 
implementation of the new assessment model for two reasons: 
(a) Norfolk Public Schools was scheduled for federal program 
monitoring in May, 1993, which included a review of 
vocational assessment compliance for special populations and 
the practicum manager was urged to step up efforts and 
produce a model for assessing all special-needs students, 
and (b) an interpretation of the federal legislation (Public 
L-:w 101-392, 1990) by congressional rulemaking committees 
indicated that assessment services had to be* made available 
to disadvantaged as veil as handicapped students; the VDOE 
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interpreted this to include all economically disadvantaged 
(including foster children and teenage mothers) , 
academically disadvantaged, limited-English-prof icient , and 
special education students who were entering a vocational 
education program . The original plan had focused only upon 
improving the delivery of services to special education 
students, which was an insufficient effort on the school 
division's part, in view of these legislative mandates. 

The practicum manager and vocational evaluators worked 
during the summer months of 1992 reviewing options and 
developing more innovative methods and procedures for 
conducting the vocational assessments of handicapped 
students. Each response to the survey of vocational 
assessment centers in Virginia (see Appendixes C and D) was 
studied in an attempt to locate additional best practices 
being utilized that we might visit. Basically, the findings 
were that vocational assessment centers across Virginia were 
using the model that Norfolk had used since 1980, with 
minor revisions. The major difference found was in the 
report format. Many divisions had abandoned the 
free-standing narrative report (six to eight pages) for a 
checklist reporting format. Two divisions were looking at 
some ^hort term prescreenings so that only those students 
who needed comprehensive vocational evaluation were sent to 
the center. 

Many models of assessing handicapped students in other 
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states were reviewed. Little information was found either 
in literature review or through telephone contacts with 
seven area states for vocational assessment of disadvantaged 
students. Some of the major models reviewed are summarized 
in Table 4 and were used in planning the new implementation 
model for Norfolk Public Schools. 
Table 4 

A Comparison of School-Based Vocational Assessment Models 
for Use with Handicapped Populations 



Location 


Number of 


Aptitude 


Informal 


CBVA 


Work 


SA 






levels 


tests 


assessment 




samples 




Illinois 




3 


No 


Yes 


Yes 


Yes 


Yes 


Kentucky 




1 


Yes 


No 


No 


Yes 


Yes 


Texas 




2 


Yes 


Yes 


Yes 


No 


No 


Pennsylvania 


1 


No 


Yes 


Yes 


Yes 


Yes 


Massachusetts 


2 


No 


Yes 


Yes 


Yes 


Yes 


Missouri 




2 


No 


Yes 


Yes 


No 


Yes 


D.C. 




3 


No 


Yes 


Yes 


No 


Yes 


Colorado 




3 


No 


Yes 


Yes 


No 


Yes 



Note : CBVA is curriculum-based vocational assessment; SA is 
situational assessment, which includes work-sites 



The Illinois Model (Hayes, Warren, and Lopez-Valdez, 
1988) was the most comprehensive assessment model reviewed 
and came the closest to matching the ideas being planned for 
design of the Norfolk model of vocational assessment. The 
Illinois Model was intended as a means to provide services 
to students from all of the identified special populations: 
handicapped, limited English proficient, academically 
disadvantaged, and economically disadvantaged. This model 
was developed by a group of evaluators and educators for the 
Illinois State Board of Education. The authors incDrporated 
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little of the traditional comprehensive vocational 
evaluation process in this model. Rather, it was process- 
based and focused on collecting data on the student from the 
programs and classes in which the student would ordinarily 
be functioning. There was not a "pull-out 11 (from classes) 
focus in the Illinois Model. The Kentucky model (Uthe, 
1980) was almost an identical match to the former Virginia 
model; it utilized a traditional work sampling approach to 
testing vocational potential. This was the comprehensive 
vocational evaluation model from which Norfolk was 
attempting to evolve through this project. It focused on 
the needs of handicapped students and did not mention the 
other special populations. 

The >T orfolk model for assessment of handicapped and 
disadvantaged students integrated many of the components 
studied from the various states. It featured a multiphase 
approach rather than the single-stage approach used in the 
previous assessment model (see Figure 3) . 

The Pennsylvania model (Minugh and Morse, 1981), 
developed through the Philadelphia School District, was 
specifically for handicapped youth, as was the Texas model 
(Edinburg Consolidated Independent School District, 1979) . 
The Massachusetts model (Stodden, 1980) was based upon the 
collection of data through work sampling and the traditional 
rehabilitation model and the integration of curriculum-based 
data collection methods. 
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F igure 3 . The Norfolk multiphase vocational assessment 
model . 

Note: The numbers represent the evaluator-to-student ratio 
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Albright and Cobb (1988(a)) developed a model through a 
3-year grant , which was later implemented in Colorado and in 
Washington, D.C. This model, according to the authors, was 
a collection of modules dealing M with the rationale of 
curriculum-based vocational assessment (CBVA) for secondary 
school students with handicaps 11 (p. 1) • The models that 
evolved from this work are anticomprehensive vocational 
evaluation and offered no solutions that utilized work 
sampling or other traditional evaluation methods. Their 
works proposed that all previous vocational efforts within 
the schools be abandoned in favor of collecting data from 
teachers, from records, and from classroom activities. This 
model was a radical departure from accepted methodology. It 
was to the 1989 oral presentations by Albright and Cobb in 
national forums that the practicum manager had responded 
with strong verbal opposition on this issue of CBVA. 
Only one other model, the one in use in Missouri 

(Maxam, 1986) , specifically identified "assessment of the 
disadvantaged, " (p. 1) so the development of a model of 
services for that population was left to the creativity of 
the practicum manager and staff. 

In the new multiphase model, assessment begins in the 
eighth grade with the testing of all students utilizing the 
Differential Aptitude Test (DAT) . As discussed in the 
manual for Project Vocational Assessment Implementation 

(Texas Education Agency, "'.982) , the DAT was selected for 
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inclusion in the Norfolk multiphase assessment model for a 
number of reasons. It could be administered either to 
groups or to individual students. Make-up testing could 
easily be provided if a student missed 1 or 2 days of the 
g; ip administration. Administration time was about 4 
hours, spread over 3 or 4 school days. Machine scoring was 
necessary because there was such a large group to be tested, 
and that was available with the DAT. This paper-and-pencil 
test measured nine aptitudes including: verbal reasoning, 
numerical ability f abstract reasoning, clerical speed and 
accuracy, mechanical reasoning, spatial relations, 
spelling, language usage, and general mental ability* Use 
of a separate answer sheet meant that test booklets 
purchased by NPS could be reused. The DAT lent itself to 
certain adaptations so that testing could include deaf, 
physically handicapped, learning disabled, emotionally 
disabled, and some visually impaired students. It had a 
sixth-grade reading level, which made it appropriate for the 
disadvantaged population. The DAT was normed for students 
in Grades 8-12. This test had previously been administered 
only to lOth-grade students, but was moved to eighth grade 
as a result of this project (see Appendix E) . 

Students who were identified as handicapped or 
disadvantaged and who demonstrated the need for further 
assessment beyond the DAT received curriculum-based 
vocational assessment. If further information was 
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necessary, students moved through a continuum of services 
(see Appendix F: Middle School Assessment model and Appendix 
G: High School Vocational Assessment Model) classified as 
Phase I and Phase II Assessments in both middle school and 
high school. 

A major component of meeting this terminal objective 
was development of a means to identify those students in 
Grades 8-12 who met the definitions provided by VDOE and 
the Perkins 1 regulations as disadvantaged, LEP, or 
handicapped. The project manager had previously developed 
methods to identify high school handicapped students. 
However, the new undertaking was much more involved. For 
example, an easy identifier of economic disadvantage was 
eligibility for free or reduced-price school lunches. 
However, other federal legislation had made that information 
confidential, and the local school district interpreted that 
to mean that such identification could not be released to 
any other department within the school division, for any 
reason. 

Two computer programmers from the Department of 
Management Information Systems (MIS) worked with the project 
manager to develop an identification system that would not 
reveal the confidential information mentioned above. The 
system ran a check of all enrolled students in Grades 8-12. 
In the first srort those students who were handicapped 
or LEP were identified. The second sort identified those 



students who had been retained for two or more times. The 
third sort listed students with grade point averages of D or 
below. The next sort added to the list those students who 
scored in the lowest quartile on the Literacy Passport 
Tests , the Iowa Tests of Proficiency, or other standardized 
tests. The final sort added the names of students on free 
or reduced-price lunches. However, only a summative list, 
which includes the names of students from all of the sorts, 
has been released to the Department of Adult and Vocational 
Education. Thus, the reason for appearing on the list is 
unknown to persons outside of MIS. 

The identification process revealed that Norfolk Public 
Schools had a total student enrollment in Grades 8-12, as 
of February 19, 1993, of 9,871 students. Of these, 6,114 
met the criteria as disadvantaged, 1,053 were identified as 
special education, and 17 students were identified as 
limited English proficient. It should be noted that LEP 
students are identified based upon self -identification at 
the time of enrollment into the school system. The total 
eligible population to be served by the vocational 
assessment process was 7,184, which represents 72.8% of the 
Grades 8-12 enrollment in the school division. The results 
of the sorting procedure are summarized in Table 5* 
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Table 5 

Identification of Norfolk Special Needs Students 



School 



Enrollment Disadv Handicap LEP Percentage 



Granby H 
Lake Taylor H 
Maury H 
Norview H 
Washington H 
Azalea Gdn M 
Blair M 
Laf-Winona M 
Lake Taylor M 
Norths ide M 
Norview M 
Rosemont M 
Ruffner M 



1572 
1456 
1611 
1589 
1278 
320 
298 
263 
269 
365 
352 
262 
236 



1018 
846 
841 

1036 
881 
229 
145 
178 
141 
214 
255 
160 
170 



152 
208 
149 
140 
154 
43 
25 
28 
35 
30 
34 
27 
28 



7 



2 
1 



6 



1 



74.8 
72.4 
61.9 
74.0 
81.1 
85. 0 
57.0 
78.3 
65.4 
66.8 
82.7 
71.8 
83.9 



Totals 



9871 



6120 



1053 



17 



72.8 



Source: Norfolk Public Schools, MIS , February 19, 1993 

The identification process confirmed the project 
manager's belief that a large proportion of the students in 
Norfolk Public Schools would require vocational assessment 
as required in the Perkins 1 regulations. The identification 
process has been established to provide data on an ongoing 
basis, so that students may be identified and assessed 
regularly. 

Terminal Objective 2 Restated and Expanded 

2. To provide vocational evaluation services to at 
least 20% of eligible special-needs students. Of the 1200 
eligible special education students, 172 were assessed by 
three evaluators in the 1991-1992 year, using the former 
model. The goal was to evaluate at least 240 special 
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education students utilizing two vocational evaluators and 
to evaluate at least 1200 (20%) of the identified 
disadvantaged students. 
Termina 1 Obi ect ive 2 Accompl ishments 

This objective was expanded from the goal in the project 
proposal of assessing , through comprehensive vocational 
evaluation, an additional 20% of handicapped students. The 
new model of assessment included in-building or Phase I 
assessments of special-needs students in Grades 9 and 10 and 
accommodated those handicapped students in Grades 11 and 12 
who were not previously evaluated. Whereas one evaluator 
could work with only thre^ handicapped students per week in 
the previous model, the Phase I model allowed each evaluator 
to complete up to 10 student assessments per week (see 
Appendix H) . In addition to this modest objective to increase 
the number of handicapped students served through this model, 
all students enrolled in eighth grade (approximately 1,400), 
including special-needs students who were academically or 
economically disadvantaged or limited English proficient, were 
to be assessed for vocational aptitudes using the Differential 
Aptitude Test. The project manager proposed continuing the 
DAT administration at 10th grade, as well, for the 1992-1993 
and 1993-1994 school years, as an assurance that no group of 
students would miss being tested. 

End-of-year results (complete as of June 18, 1993) 
showed that 342 vocational evaluations (Phase I and Phase II) 

69 




were conducted during the 1992-1993 school term. That 
compared to the 172 students evaluated during the 1991-1992 
term, showing an increase in students evaluated of 50.3%. 
That increase was despite the loss of one vocational evaluator 
position and can be attributed to the new innovations. This 
assessment of 29% of the total special education secondary 
population exceeded the project r x l of assessing 20%. 
End-of-year data are further analyzed in Appendix I — 
Population Analysis: Norfolk Public Schools Vocational 
Evaluation Center. 

In addition to the assessment services provided to 
handicapped students, the DAT was administered to 2058 middle 
school students and 1333 high school sophomores (see Appendix 
J — Differential Aptitude Test Summary) . The total number of 
students assessed with the DAT was 3391; of that number, 2509 
were special-needs students who had been identified using the 
MIS procedures. That meant that 74% of those students taking 
the DAT were special-needs students. The assessment of these 
2509 disadvantaged students far exceeded the goal, which was 
to assess 1200 disadvantaged students (20%) . 

Of the 9871 students enrolled in Grades 8-12 in NPS, 
3733 were vocationally assessed during the 1992-1993 school 
year. That reflects that 37.8% of the student enrollment was 
assessed. 

Process Objective 1 Restated 

1. Critical competitors to comprehensive vocational 
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evaluation were identified and reviewed to determine 
appropriateness for inclusion in the Norfolk Public Schools 
program (see Table 4) . 
Process Objective 1 Accomplishments 

Critical competitors, which were identified from the 
review of literature, included methods that were both formal 
and informal. Some formal assessment techniques included 
pencil-and-paper aptitude testing, psychometric testing, and 
work-site-based measures related to task analysis. Some 
informal assessment techniques included observations by career 
counselors, specific classes developed for career exploration 
with documentation of abilities by the teacher, observations 
by vocational evaluators of students in regular vocational 
classes , records review for a portfolio-style assessment of 
student's abilities, and curriculum-based vocational 
assessment whereby each teacher submitted information on the 
student's performance in class and an evaluator drew relevant 
vocational conclusions. Each of these critical competitors 
received consideration in the development of the Norfolk 
vocational assessment model. 
Process Objective 2 Restated 

2. A survey of secondary special education teachers 
was conducted to determine their perceptions of the vocational 
assessment program and the innovations (see Appendixes K and 
L for the survey and results) . 
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Process Objective 2 Accomplishments 

The initial project proposal suggested that a survey of 
special education students would be conducted to determine why 
students refuse to attend Madison Career Center for vocational 
evaluation. After talking with guidance counselors, school 
psychologists, and special education teachers, it was 
determined that a formal survey of special education students 
would not be conducted. Students do not like to be 
identified, the project manager was told, and response to a 
formal instrument was predicted to be quite low if students 
found out that only special education students were being 
surveyed. Instead, special education students were 
interviewed as part of the Phase I evaluation process during 
January and February 1993, in each of the five high schools. 
Students reported a favorable perception of in-building 
vocational assessment. Most of the students in mainstreamed 
or co-teaching classes continued to refuse a comprehensive 
vocational evaluation at Madison Career Center- even when they 
were told that it would provide useful information that would 
help them get into higher level vocational classes. 

A formal survey was conducted with high school special 
education teachers (see Appendixes K and L) . A total of 64 
surveys were distributed with a return rate of 53%. The 
survey and the tally of responses confirmed the practicum 
manager 1 s belief that the new model must offer an array of 
options for the collection of vocational assessment data. Of 
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those responding, 90% concurred that special education 
students needed more than pencil-and-paper testing, 
Familiarity with the 4-day comprehensive vocational evaluation 
was reported by 93% of those responding, which was a 
disappointment after the intensive efforts of the introductory 
practicum (Feldt, 1992) . Only 30% of respondents had visited 
the evaluation center at Madison, 43% felt that the evaluation 
took too long, and only 50% reported that their students could 
afford to miss 3 or 4 days from classes in order to be 
evaluated. 

The practicum manager looked at utilization of the 
results of the vocational evaluation based upon the survey 
data. Although 90% of teachers reported that they used the 
vocational evaluation results when developing the student's 
transition plan, only 10% reported that they had ever included 
the vocational evaluator in the development of that plan or 
the Individualized Education Plan (IEP) . Eighty-three percent 
of the teachers responding reported that they read the entire 
vocational evaluation report, not just the recommendations* 

The overall results of the survey may be skewed somewhat 
because the respondents did not equally represent all areas of 
instruction within special education. The data are analyzed 
to illustrate which groups of teachers responded, by school 
(see Tables 6 and 7) . However, the interpretations of the 
data expressed above were based on the composite of responses, 
not on a per school basis. 
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Table 6 



Representativeness of the Response Group to Surv ey of High 
School Special Education Teachers 



School 


Surveyed 


Returns 


Subiect Tauqht bv 


Respondent 


Code 


N 


N 


% 


ED EMR 


LD-SC LD HI TMR BD 


MH SPH 


01 


10 


4 


40 


2 




2 






02 


12 


8 


67 


2 


1 


4 




1 


03 


11 


9 


82 


2 2 


1 


4 






r\ a 
04 


13 


4 


31 


1 




3 






05 


18 


9 


50 


2 




2 1 


2 


1 1 


TOTALS 


64 


34 


53 


9 2 


2 


15 1 


2 


1 2 


Table 7 


















Response 


Rate by 


School and 


by Subiect 


Tauqht 
























School 


Response 






Subiect Taucrht 




Code 


Rate 


ED 


EMR 


LD-SC 


LD 


HI TMR 


BD 


SPH 


01 


40% 


100% 


0 


0 


33% 








02 


67% 


100% 


0 


100% 


67% 




0 


100% 


03 


82% 


100% 


100% 


100% 


67% 








04 


30% 


50% 


0 


0 


43% 




0 




05 


50% 


100% 


0 


0 


40% 


100% 50% 100 


% 100% 


Process 


Obiective 


3 Restated 











3. An experimental group of students from Granby Kigh 
School, who did not go through the comprehensive vocational 
evaluation process, but who had a curriculum-based assessment 
or a Phase I assessment, were monitored to determine whether 
they were admitted to the recommended vocational training 
programs, and their grades and attendance in vocational 
courses were monitored as indicators of success in vocational 
programs. These results were then compared to the same 
information on a control group of similar students who had 
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completed a comprehensive vocational evaluation prior to 
entering vocational training programs (see Figures 1 and 2) . 
Process Objective 3 Accomplishments 

After 15 students at Granby High School were assessed 
using the Phase I (records review) and curriculum-based 
vocational assessment components of the new model, another 
group of students with v.he same handicapping condition and 
similar special education program placements was selected from 
among those students who received comprehensive vocational 
evaluations during the 1991-1992 school term. The identities 
of the 15 students in the experimental group and the 15 
students in the control group were made known to the 
vocational resource teachers at the home sc/iools, L>ut they had 
no way of knowing which group the students fell into, only 
that follow-up data were requested. Follow-up data were 
collected to illustrate vocational enrollments of these 
students . 

As shown in Figures 1 and 2, those students completing a 
center-based vocational evaluation were much more likely to be 
recommended for higher level vocational courses such as the 
programs offered at Norfolk Technical Vocational Center (NTVC) 
than students receiving a Fhaso I assessment. Eighty percent 
of the control group were recommended for NTVC compared to 
66.6% of the experimental group. On the other hand, none of 
the students who received Phase II comprehensive vocational 
evaluations were recommended for vocational training programs 



within the home school , but 6.7% of the experimental group 
were. Students in the control group were able to tour Madison 
Career Center (MCC) while there for evaluation, resulting in 
13 . J% of them being recommended for training at that site. No 
students from the experimental group were recommended for MCC. 

When outcomes were compared at the end of the 1992-1993 
school term, it was noted that 13.3% of the students in each 
of the groups dropped out during the school year and 6.7% of 
each group moved out of the school district, resulting in a 
loss of 20% of the experimental and control group populations. 
Of the control group 80% had been recommended to attend NTVC, 
but only 3 3.3% had done so. Of the experimental group 66.6% 
had beer recommended for NTVC, but only 2 0% attended. The 
outcome th*t was observed but had not been recommended was 
that so many students had remained in their home schools for 
vocational training (control group 40%; experimental group 
13.3%) and that 33.4% of students in the experimental group 
were taking academic classes only, with no vocational program 
enrollment, as a result of having failed one or more classes 
during the previous year. 

Success in the vocational programs was demonstrated by 
students 1 grades in vocational classes and students' 
attendance. The experimental group fared better (see Table 
8) . 
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Table 8 

Comparison of Attendance and Grades in Vocational Programs 



for Students in the Control and Experimental Groups 



Control 



Experimental 



Passing vocational 
program 



7 



4 



Failing vocational 
program 



4 



0 



Average days absent 



16 



10 



Process Objective 4 Restated 

4. A survey of school-based vocational evaluation 
centers in Virginia was conducted as one step in the 
identification of alternative evaluation strategies (see 
Appendix C for the survey and Appendix D for results) . 
P rocess Objective 4 Accomplishments 

Mailings were sent to the superintendents of all school 
divisions in Virginia, asking for the identification and 
address o^ the vocational evaluators for their district. 
Responses were received from 100% of the districts' 
superintendents. Formal surveys were then mailed to the 
identified evaluators (see Appendix C) . The responses were 
tallied (see Appendix D) and shared with evaluators across 
Virginia in October, 1992, at a statewide workshop held in 
Spotsylvania County, Virginia. 

Evaluators in 13 centers responded to the survey. 
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The average number of evaluators was two per center; 10 of the 
centers had secretarial support for report preparation. 
Eleven of the centers responding reported that they developed 
their reporting format based upon the original reports used in 
Virginia (prior to 1980) . Productivity was reported at 
between 75 and 100 students per evaluator per year. The 
methods of assessment and procedures reported by the 13 
centers responding were described as those in use in NPS 
before the implementation of the project interventions. No 
alternative methods of assessment were identified through the 
survey. 

Process Objective 5 Restated 

5. Vocational evaluation personnel reviewed 
alternatives to the former method of obtaining vocational 
assessment data. Through use of materials from the literature 
review, site visitations and surveys, the evaluators and 
program manager redesigned the Norfolk vocational evaluation 
model, updating it from the existing 4-day pull-out model, 
which had been initiated in 1980. 
Process Objective 5 Accomplishments 

This objective was accomplished, but required hundreds of 
teanhours, some of which required the evaluators to work during 
their noncontract months. Efforts are summarized above in the 
literature review and in the description of accomplishment of 
the first terminal objective. The result was the development 
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of the Norfolk Multiphase Vocational Assessment Model, 
illustrated in Figure 3. 
Process Objective 6 Restated 

6. Alternative approaches to vocational evaluation were 
implemented; the comprehensive vocational evaluation component 
was maintained, but fewer students were found to need such 
in-depth assessment . 

Process Objective 6 Accomplishments 

This objective was accomplished as stated. The impact of 
the interventions on the assessment of special-needs students 
was described above in the accomplishments of Terminal 
Objectives 1 and 2. The comprehensive evaluation process 
became the Phase II component of the Norfolk Multiphase 
Vocational Assessment model. It was kept available for 
students who needed further exploration and for those whoso 
handicapping conditions required reasonable accommodations in 
the training or work aite. However, it was used for fewer 
students; as shown in Appendix I, 111 students completed 
comprehensive or Phase II assessments in 1992-1993, compared 
to 172 the previous year. 
Process Objective 7 Restated 

7. Observations were conducted of the vocational 
resource teachers as they explained the vocational evaluation 
process and as they interpreted the findings to students, 
parents, and special educators. 
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Process Objective 7 Accomplishments 

The practicum manager observed the vocational resource 
teachers at all five high schools in meetings with students 
and in meetings with students, parents, and special education 
teachers (and even community agency representatives) . They 
have all demonstrated the ability to interpret findings in 
terras of courses to be scheduled, IEP modifications to be 
developed , and reasonable accommodations to be expected • No 
deficits have been documented in any case. 
Process Objective 8 Restated 

8 . The results of the various vocational assessment 
processes and methods were reviewed, revised as needed, and 
recommendations for changes have been made. 
Process Ob j ect i ve 8 Accompl ishments 

This objective was one of the last ones to be 
completed. Evaluators had an unencumbered week at the end of 
the school year to collect and analyze student population data 
and to create plans for the 1993-1994 school year. They were 
offered an open forum to express their reactions to the 
interventions and to list praises or criticisms. Each 
component of the new multiphase assessment model was examined 
separately in an effort to develop revisions. 

The one overwhelming concern was that there was not 
sufficient time to meet all the demands being placed upon the 
vocational evaluators. The less experienced evaluator ended 
the school year with 4 0 unwritten reports on the students she 
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had assessed. Time management and organization need to be 
stressed as operational tools for this individual during the 
next year. 

In addition, arrangements have been completed in 
conjunction with computer programmers from MIS to put all 
vocational assessment reporting on the school division 
mainframe network, known as the Student Information System 
(SIS) . This will alleviate the need for written reports on 
most students by providing instant access to a checklist of 
the recommendation options . The computer screen designed for 
the SIS Vocational Assessment report is shown in Appendix Q. 
Hardware has been requested, training has been scheduled, and 
funds have been requested to pay staff to be trained to work 
on the SIS over the summer (August 1993) . This effort should 
make results of the vocational evaluations more accessible by 
counselors and teachers as they work with students. It will 
also be available to the principals at NTVC and MCC and will 
help them to admit stuients based upon the results of the 
vocational assessments. Differential Aptitude Te.s- (DAT) 
results have been placed cn the system for students tested 
during 1992-1993, because it is available from the scoring 
service on magnetic tape. Input of all other assessment 
results will be manual and time consuming, but has been 
planned as part of the schedule for 1993-1994. The DAT will 
be administered to both 8th- and lOth-grade students again in 
1993-1994, but will be completed earlier in the school year. 
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Plans were made to share the burden of visiting all middle 
school classes to interpret the results of these tests. 
Three other central office vocational educators have been 
proposed to share this responsibility with the project manager 
for the 1993-1994 year. A method of follow-up will have to be 
devised for those students who miss taking the DAT or whose 
results are invalid or missing. This will be addressed in 
meetings with guidance counselors during September 1993. 

Phase I assessments will continue next year as 
structured. The evaluators suggested that they be assigned to 
specific schools rather than each of them rotating through the 
five high schools as they did in 1992-1993. This will be 
discussed further , but the project manager hesitates to make 
such assignments because familiarity with teachers and 
students may have an effect on the evaluator's objectivity in 
making student recommendations. 

The curriculum-based vocational assessment component 
(CBVA) will also continue in use because it was reportedly 
helpful. High school vocational teachers commented to the 
project manager that they wanted to be asked for input: on 
students' performance in their vocational classes , which was 
considered positive compared to the previous complaints from 
the middle school vocational teachers. A CBVA teacher 
questionnaire was piloted , reviewed, revised, and 
redistributed as part of the project interventions (see 
Appendix R) which will be utilized during 1993-1994* 
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Comprehensive evaluation or Phase II assessments will 
also be offered without change during the next school term, 
with the exception of the computerized SIS student reports 
rather than formal, written narrative reports. 

The evaluators were most enthusiastic about the 
innovations tried during 1992-1993 related to the 
community-based and enclave site-based assessments of certain 
students. They accompanied students to area hospitals, 
department stores, and other business sites and documented 
work behaviors and skills in those environments, Although 
time-consuming, it was recommended that this method be 
continued for students in ths community placements. 

The evaluators recommended the development of a 
carbon-pack form to allow them to provides; immediate feedback 
*tc the home schools on certain medical needs {e.g-, eye 
examination or doctor ? s release for vocational education) and 
for other special needs such as counseling or crisis 
intervention. They expxessed concern that, when reports are 
put into SIS rather than mailed back to the referral sources, 
these needs may go unnoticed and unmet. They have developed 
a draft of such a document and will prepare it for printing in 
September 1993 • 
P rocess Objective 9 Restated 

9. Staff development activities were conducted to 
inform vocational resource teachers of the new vocational 
assessment procedures and to disseminate copies of the newly 



developed model. 

Process Objecti ve 9 Accomplishments 

Staff development activities were conducted during the 
city-wide staff development day in October 1992. Resource 
teachers received copies of the new model of assessment (see 
Figure 3 and Appendixes F, G, and H) along with an explanation 
of each component. At the regularly scheduled monthly 
departmental meetings held by the project manager, updates 
were provided and questions were answered. 

In one meeting, for example, resource teachers asked why 
they were asked to photocopy materials from student's 
confidential files and send theiu to the vocational evaluators, 
for those students who were scheduled to have a Phase II 
assessment. This had been the requirement since 1980. 
However, in view of the new interventions, which put the 
vocational evaluators in the school houses 2 or 3 weeks out of 
each month, it seemed reasonable that they could review 
records on those students who required further assessment. 
This eliminated the photocopying task for the vocational 
resource teachers. As described in Process Objective 7 , 
above, no deficits were observed in the resource teachers 1 
understanding of the vocational assessment process or their 
interpretations of the results to other school personnel. 

The support of the vocational resource teachers is 
evident in the year-end results, because they scheduled the 
evaluators to be in their schools, referred students, and 
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coordinated collection of CBVA data for those students 
assessed. They will continue to serve these roles in 1993- 
1994. 

Process Objective 10 Restated 

10. Staff development activities were conducted so the 
vocational evaluators could thoroughly implement all new 
procedures . 

Process Objective 10 Accomplishments 

The two evaluators on staff worked additional weeks 
during July and August 1992, to review existing models and to 
develop plans for the new assessment model, and in August 1993 
to revise the handbook and develop staff development plans for 
guidance counselors and vocational department chairmen. They 
met weekly with the practicum manager to plan, to develop the 
new model, to meet with other staff, to devise implementation 
strategies, to develop a shorter vocational evaluation report, 
to redesign the brochure used by the school system to explain 
vocational assessment services to parents and the public (see 
Appendix M) , and to begin writing a nnw operations manual. 

The practicum manager arranged staff development 
activities on an individual and group basis for the evaluators 
and provided individual observations and follow-up. There was 
disagreement on the steps evaluators should take when 
completing a Phase I (records review/ interview) assessment at 
the home schools. The practicum manager and evaluators 
collaborated to develop a workable procedure (see Appendix N) , 
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which all agreed to follow when conducting Phase I 
assessments. The evaluators attended two state conferences on 
vocational assessment of special populations, Spotsylvania 
County, Virginia in October 1992 and Henrico County, Virginia 

in February 1993. In addition, they attended the national 
Issues Forum of the Vocational Evaluation and Work Adjustment 
Association, May 1993, in Virginia Beach. This was the first 
time that more than one staff development activity per year 
has been approved for the vocational evaluators. 

The practicum manager initially reviewed all vocational 
evaluation reports that were completed using the new 
assessment model. Finding only minor corrections, this 
process was limited to the review of a monthly sampling of 
reports. 

Because the project manager and the vocational evaluators 
have not previously worked on the SIS system, additional 
training has been scheduled for August 1993. Once the staff 
have been trained they will be given access codes, allowing 
them to input data on students into the system. 
Process Objective 11 Restated 

11. Staff development activities were conducted with 
administrators , vocational educators , specia 1 educators , 
guidance counselors, school psychologists, and school-based 
rehabilitation counselors to assure an understanding of new 
vocational assessment procedures. Role definition related to 
the new process was stressed. 
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Process Objective 11 Accomplishments 

Staff development activities had been planned as formal 
activities , scheduled with groups of school division 
personnel. That could not be arranged because the activities 
had not been scheduled and approved prior to May 1992, when 
the schedule for the 1992-1993 year was sent out. However , 
the project manager was able to meet in small groups with 
building level administrators, special education 
administrators, vocational education administrators, and 
school psychologists. Two staff development activities were 
conducted with high school guidance counselors, one of which 
they sponsored and invited the practicum manager and her 
supervisor to lunch. 

Middle school guidance counselors received group 
instruction on administration of the Differential Aptitude 
Tests to eighth graders and on scheduling the practicum 
manager to meet with the students in order to provide 
interpretation of the results* The students were tested in 
December 1992, and test results were returned in January 1993. 
Middle school guidance counselors were notified to contact the 
practicum manager before distributing test results (see 
Appendix 0) . Each counselor had developed an interpretation 
method that was best suited to their school and students J 
schedules, and the project manager scheduled all visits 
according to those requests (see Appendix P) • 

The practicum manager provided to small groups of middle 
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school students an explanation of the DAT Student Profile 
Sheet, which was returned to the school after scoring by The 
Psychological Corporation, and provided a copy for the student 
to take home, with the original going into the students' 
cumulative school files. Middle school counselors attended 
these interpretation sessions so they could understand the 
interpretation given to students and so that they might 
conduct them for their students as needed. 

A meeting was held in November 1992, under the direction 
of the Coordinator of School Psychologists with NPS, for the 
project manager to explain the multiphase assessment model to 
the secondary school psychologists and to the school case load 
counselors from the Virginia Department of Rehabilitative 
Services. Some technical questions related to the testing of 
concrete thinkers with abstract methods, and the project 
manager addressed them based upon readings from the literature 
review. School psychologists were asked to document academic 
performance levels on each student they evaluate as this 
information is essential in making vocational predictions. 
They agreed to include collection of this information in their 
testing procedures. The psychologists and the project manager 
will meet in 1993-1994 to discuss the sharing of information 
and the possibility of adding the vocational assessment as a 
component of each secondary student's triennial evaluation. 
Process Objective 12 Restated 

12- A summary of activities, implementation efforts, and 
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findings were developed into a paper, presentation, and 
handouts and were presented to vocational evaluators at the 
Virginia conference, May 1993 . Composite results of the 
statewide evaluation center survey were provided to all sites 
responding. The project manager presented at the Virginia 
Vocational Special Needs state conference, August 1993 and 
will speak at both the International Conference of the Council 
on Learning Disabilities, October 8, 1993, and the 
International Conference of the Council for Exceptional 
Children, October 15, 1993. 
Process Objective 12 Accomplishments 

Handouts were developed as the multiphase vocational 
assessment model was implemented. As findings were determined 
and outcomes documented, a formal presentation was readied. 
The practicum manager has also been asked to submit a proposal 
to present at the Seventh Forum on Issues in Vocational 
Evaluation and Work Adjustment to be held in March 1994 and 
was selected to present at the International Conference of the 
Council for Exceptional Children and at the International 
Conference of the Council for Learning Disabilities. In 
addition, she has been asked to prepare journal articles for 
the Journal for Vocational Special Needs Education and The 
Vocational Evaluation and Work Adjustment Association (VEWAA) 
Bulletin . 

Results of the statewide survey of school-based 
vocational evaluators (see Appendix D) were compiled, copied, 
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and distributed in October 1992, at the VEWAA Regional 
Workshop conducted in Spotsylvania, Virginia. Additional 
copies were made available at the spring regional workshop 
conducted in Henrico County, Virginia in February. 
Process Objective 13 Restated 

13. A statewide clearinghouse of surplus vocational 
evaluation equipment was developed. 
Process Objective 13 Accomplishments 

When this project was initiated, much support was 
provided by evaluators in school-based facilities for the 
development of a statewide clearinghouse of surplus vocational 
evaluation equipment. Indeed, the results of the survey of 
vocational evaluators in Virginia (see Appendix D) revealed 
that most facilities had work samples or other commercially 
developed evaluation systems that were not being utilized. 
Many expressed an interest in selling or trading these items; 
others were uncertain whether these items could leave the 
school district because they were purchased with restricted 
funds. The results of the survey, including listing of 
surplus materials was distributed to assessment centers in 
Virginia. 

However, after researching the various models and 
redesigning the assessment model for Norfolk, it became 
apparent that the trend in school-based assessment must be 
away from pull-out testing and exploration, in favor of less 
intrusive methods, like those implemented in the NPS model. 
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The practicum manager initiated plans to share the surplus 
work samples with occupational exploration classes for raidUle 
school students for use as a training rather than testing 
station. Those work samples that were normed on mentally 
retarded students were placed in the sheltered workshop and 
the classes for trainable mentally retarded students, again as 
a training tool, such as the development of work speed, 
improvement of tool usage, and for work hardening. 
Process Objective 14 Restated 

14. Project activities were conducted, including the 
collection and analysis of data, regular meetings with 
school division project committee, and development of progress 
reports at 6-month intervals. 
Process Objective 14 Accomplishments 

Throughout the period March 1992, until completion of the 
activities related to the terminal and process objectives, the 
project manager followed the structure provided in the time 
line. This and the maintenance of a journal of activities 
served as guides and as motivators to keep the project moving 
forward. Interim reports were developed in draft form and 
were reviewed in individual conferences with each of the three 
members of the local project committee. Concerns and changes 
expressed during those conferences were reflected in the 
interim reports before they were submitted to the university 
committee. The project activities ware concluded in July 
1993, but some areas related to the project will continue into 
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the next school term. 
Summary of Results Related to Terminal Objectives 

This project was conducted to fulfill two terminal 
objectives. These were met through the accomplishment of 14 
process objectives. Evidence of successful completion is 
provided in Table 9 and Table 10. 

1. To develop alternative methods for conducting 
vocational assessment of secondary special-needs students, 
including those eligible for special education services, those 
who are economically and academically disadvantaged, and those 
with limited English proficiency; to explain the new methods 
to school division personnel; to implement the alternative 
methods; and to evaluate the methods for effectiveness and 
efficiency. 
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Table 9 



Evidence of Completion of Terminal Objective 1 



Activity 



Outcome 



Reviewed literature 



Surveyed other Virginia 
school districts 



Surveyed special educators 

Developed method of identifying 
special-needs students 



Developed multiphase 
vocational assessment model 



Monitored progress of control 
group and experimental group 



Developed listing of 
potential methods and 
other working models 
(best practices) . 
Identified gap between 
best practices and 
state's vocational 
evaluation model. 
Identified concerns and 
student needs. 
Developed listing by 
school, in alphabetical 
order of handicapped, 
disadvantaged, and 
limited English 
proficient students; used 
as a guide to develop 
schedule and new model. 
See Appendix F, Appendix 
G, and Figure 3 ; held 
staf f development for 
guidance counselors , 
vocational evaluators, 
vocational resource 
teachers, school 
psychologists , and 
administrators. 
No major difference in 
student outcomes 
regardless of method of 
vocational assessment. 



2. To provide vocational evaluation services to at 
least 20% of eligible special education students. Of the 1200 
eligible special education students, 172 were assessed by 
three evaluators in the 1991-1992 year using the former model. 
The goal was to evaluate at least 240 special education 
students by two vocational evaluators and to evaluate at least 
1200 (20%) of the identified disadvantaged students. 
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Table 10 

Evidence of Completion of Terminal Objective 2 



Activity Outcome 



Conducted Phase I assessments of 
special eaucdLion suuaenus 


Evaluated 


2 31 students 


Conducted Phase II assessments 
of special education students 


Evaluated 


111 students 


Tota 1 special education 
students evaluated 




342 


LonaucteQ uitierenuiai Aptituae 
Testing with all eighth graders 


Evaluated 


2 058 students 


Conducted Differential Aptitude 
Testing with all 10th graders 


Evaluated 


1333 students 


lotai SLUuerits receiving DAT 




3391 


Identified disadvantaged 
students taking DAT 


Evaluated 


O") CQ em +• 1 i /—\ 4" em 

z s oy Stuaerius 


Identified limited-English 
proficient-students taking DAT 


Evaluated 


6 students 


Total disadvantaged students 
receiving DAT 




2365 


Data Analysis Procedures 






The data from the surveys were analyzed 


based upon 


percentages of responses to various queries ♦ 


Student 



anecdotal responses from the interviews were analyzed in 
narrative format. Data on student identification were 
provided by the Management Information Systems (MIS) office of 
the school division, based upon information in their data 
base* 

The MIS personnel cross-tabulated information on students 
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to develop an identification of students who met the federal 
and state definitions of special populations. They compared 
students to eligibility criteria and presented a list by 
school, in alphabetical order by grade level, based upon a 
check of students' grade-point averages, handicapping 
conditions, test scores (to determine those performing in the 
lowest quartile) , limited-English-proficiency identification, 
and free or reduced-price lunch eligibility. This 
identification of students was invaluable in attempting to 
meet the criteria set down in the legislation (Public Law 
101-392, 1990). 

Data have been analyzed to determine the numbers of 
special education students who were evaluated and the number 
that enrolled in vocational education classes. This analysis 
showed that special-needs students were not excluded from 
enrollment in vocational classes. In fact, the overall 
enrollment of special needs students in vocational education 
courses was very high; certain vocational classes showed an 
enrollment of 98% disadvantaged students. The analysis 
demonstrated that efforts needed to be focused on providing 
assessment to larger numbers of students and especially on 
providing assessment to disadvantaged students. This 
population was shown to be severely underserved in the 
previous model, but well addressed by the multiphase 
assessment model. 

95 



ERLC 



103 



Results of Strategy Activities 

The solution strategies appear to have been appropriate 
and have been most effective in improving the delivery of 
vocational assessment services to special-needs students in 
Norfolk Public Schools. The need was to develop and implement 
a new m^del of vocational assessment in order to allow more 
handicapped students access ^:id to provide the same 
opportunities to disadvantaged students. This has been 
accomplished, terminal objectives have not only been met, but 
exceeded, and the process objectives were completed as stated. 
Discussion 

In conducting these project activities, including the 
review of vocational assessment methods in other states, the 
project manager met and interviewed almost a dozen of the 
acknowledged leaders in the field of vocational evaluation. 
Each of them offered support, information on their efforts, 
articles or texts of procedures manuals, and a request for the 
results of the innovations being implemented in Norfolk. 
Their encouragement for the project manager to use proven 
methods in conjunction with new strategies was motivating and 
reassuring. 

The project manager has already been invited to present 
the project results on October 8, 1993, at an international 
conference in Baltimore, Maryland and at a national conference 
in Albuquerque, New Mexico, on October 15, 1993. 
Dissemination is being facilitated by these other 
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professionals who see the project innovations as timely and in 
demand. 

Results of the statewide survey of vocational evaluation 
centers conducted through this project and a review of the 
literature supported Nolte's (1989) assertion that the scope 
of vocational assessments in schools varied from state to 
state, with lack of standardization due to nonspecific federal 
requirements. However, an alarming amount of standardization 
within the state was discovered. 

The school-based vocational evaluations had not been developed 
based upon the local environment or student needs, but 
appeared to be duplicates of the rehabilitation model 
developed in Virginia in the 1970s. 

Development of the multiphase vocational assessment model 
was the result of accepting Cobb and Larkin's (1985) proposal 
M to eliminate the term vocational evaluation as it pertains to 
the entire range of assessment activities associated with 
screening, placement and program planning and monitoring for 
an individual child," maintaining that we should "assess 
individuals and evaluate programs" (p. 3) . They had maintained 
that the rehabilitation model of assessment did not adapt well 
to the educational environment because it attempted to make 
predictions about employment suitability, but not about 
curricular options, and this was found to be the case in 
Norfolk Public Schools. Report formats were revised, deleting 
recommendations for specific job placements and job codes. 

97 

105 



Instead course codes were substituted in the recommendations. 
Ancillary and medical services were retained in the 
recommendations section for those students who require support 
and individualization of instruction. 

The staff embraced the concepts put forth by Emery (1984) 
that the emphasis for providing assessment services in the 
school setting had shifted to career development and placement 
in vocational programs, rather than on placement into 
employment. The emphasis in the multiphase assessment model 
developed through this project is on student career 
development and attempts to identify what students can learn 
while in the educational setting. 

The new multiphase model of vocational assessment easily 
resulted in the assessment of larger numbers of handicapped 
students; in fact, the increase was 55.5% over the previous 
year, despite the loss of one vocational evaluator. In 
addition , 2 509 disadvantaged , handicapped , and 
limited-English-proficient students received vocational 
assessments using a pencil-and-paper method, the DAT. This 
population had not been previously served through the 
vocational assessment program. 

Perhaps the most revealing aspects of this project were 
the student outcomes based upon tracking of a control group 
and an experimental group of students. Regardless of the 
method of assessment, the majority of students were 
recommended for occupational programs at Norfolk Technical 
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Vocational Center or NTVC (30% and 66.6%;. However, only 
about one-third of students recommended for enrollment at NTVC 
actually took classes there. 

In addition, vocational assessment methods, including 
vc ,at\onal guidance, and the availability of work-relevant 
r -or lendations did not seem to have a measurable effect upon 
tiu drop-out rate of handicapped students. Both the control 
group and the experimental group lost 13.3% of students who 
dropped out during the course of this study. 

A major contrast was that students who completed a 
vocational assessment using the new innovations were more 
likely to be recommended for vocational education options 
within their home schools (6.7% compared to 0% in control 
group) and were more likely to followup by enrolling in the 
recommended classes. 

A surprising outcome of this project was the amount of 
success experienced on the lowest level of vocational 
assessment, the Differential Aptitude Test (DAT), by students 
who were disadvantaged, limited English proficient or 
handicapped. Approximately 3 0 students per school did not 
receive recommendations on the DAT, but most of these were due 
to the students absence for one or more days of the testing 
and certain information thus being missing. However, the vast 
majority of students received recommendations that can be 
used to plan high school programs of study, including 
occupational courses . 
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An unexpected outcome that helped in the rapid 
implementation of the eighth-grade testing component of the 
model was the willingness of guidance and testing personnel to 
add the Differential Aptitude Test at the middle school level 
and to pay for the cost of testing and scoring. The guidance 
counselors were eager to assist in the administration of the 
tests, to be trained in interpreting the test score profiles, 
and to collaborate with the vocational evaluators about the 
needs of specific students. The high level of cooperation and 
collaboration between departments on behalf of the students 
had not been anticipated by the project manager. 
Conclusions and Recommend a tions 

Consideration should be given to continuing the 
multiphase model of vocational assessment, as implemented and 
refined, as the means of vocationally assessing NPS students. 
However, three areas are recommended for further study and 
possible change. 

First, other methods of interpreting the results of the 
Differential Aptitude Tests to students should be devised. 
The practicum manager met with 2 058 students to explain the 
test and to interpret the results. This was extremely time 
consuming. Large group methods were shown to be least 
effective in disseminating this ir formation. Because the 
practicum manager has other work and supervision 
responsibilities, it is doubtful that this amount of time can 
be devoted to DAT interpretation in the future. 
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Secondly, a means should be developed to assure that 
students whose DAT results were inconclusive or incomplete 
have the opportunity to complete other vocational aw~?ssment 
activities in the ninth grade. The project manager developed 
a journal of those students whose results were questionable or 
who did not complete the DAT. However, the manual tracking of 
students from middle school to high sc xdI will require 
numerous additional hours, and then follow-up with the student 
and guidance counselor will be needed. The project manager 
will review the possibility of adding the need for further 
assessment onto the student's data screen in the Student 
Information System (SIS) component of the Management 
Information System. 

Finally, an area for further exploration is the inclusion 
of the vocational assessment as a component of the triennial 
special education assessment. The Triennial Integration Model 
discussed by Levinson (1989) would assure that all special 
education students received a vocational assessment. This 
idea seems to have merit and will provide ongoing assessment 
of handicapped students in middle school and high school. 
Such updated results will assist special educators in 
complying with their regulations to provide transition 
services to handicapped students. The practicum manager will 
meet with the director of special educational services and 
department staff to develop methods of accomplishing this 
goal. 

101 



ERIC 



109 



A spin-off activity was the development of a statewide 
clearinghouse of surplus vocational evaluation materials and 
equipment. This may be of help to smaller school divisions 
who are attempting to conduct vocational assessments and to 
other divisions facing budgetary restrictions. 

The public relations and promotion aspect of this project 
needs emphasis in the upcoming year. Efforts should be 
directed toward informing individuals both internal and 
external to Norfolk Public Schools of the multiphase model and 
of the benefits for students of offering several options in 
the vocational assessment arena. 

This project has resulted in the development and 
successful implementation of a new model of vocational 
assessment, which provides multiple options for handicapped 
students and which includes options for students in other 
special populations. in addition, it resulted in the 
development of a computerized system for identifying 
special-needs students and uses the mainframe network to 
provide systemwide access to each student's vocational 
assessment recommendations, as previously requested by 
counselors and administrators. 

This practicum has shown that a school division can 
improve student assessment services, even with limited 
resources. The project's initial success and recorded gains 
were due to cooperation and collaboration between departments, 
a willingness of the staff to experiment with new methods, the 
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support of administrators across the school division, and 
input from experts in the field across the country. 

In this process, Norfolk Public Schools has realized an 
increase of students vocationally assessed from 172 (1991- 
1992) to 2,400 students in 1992-1993. This was accomplished 
by better utilizing the existing staff and student records 
that were already available. The only increased cost was 
$5.00 per student for scoring of the Differential Aptitude 
Tests. The new procedures implemented as part of the 
multiphase model are in place for 1993-1994. The 
eighth-grade administration of the DAT is scheduled for 
October, 1993. Staff efforts are being directed toward 
additional paperwork reduction by recording student outcomes 
and recommendations on the systemwide computer network. That 
innovation will revolutionize the way results of the 
assessments are utilized within school divisions. 
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Appendix A 



Revised Project Time Line 



ACTIVITY 


PROJECTED 
DATE 


ACTUAL 
DATE 


Meet with vocational evaluators 
to discuss innovation 


2/15/92 


3/2/92 


Meet with vocational resource 
teachers to discuss innovation 


2/15/92 


3/11/92 


Meet with special education 
coordinator to discuss 
innovation 


3/3/92 


3/11/92 


Mail postcards to all school 
division superintendents in VA 


June 
1992 


3/13/92 


Meet with vocational evaluator^ 
to review literature for other 
innovative approaches to 
vocational student assessment 


July 
and 

August 
1992 


7/28/92 
through 
7/31/92 


Each evaluator will develop a 
specific method, concept, or 
strategy as a result of the 
review of other methods in VA 
localities, literature, local 
labor market needs, and a 
sampling of student records 


July 
and 

August 
1992 


7/28/92 
through 
7/31/92 


Establish a control group of LD 
students who completed a voc. 
evaluation and who will enter 
vocational training in 1992-93 


July and 

August 

1992 


7/28/92 


Conduct surveys of all school- 
based vocational evaluation 
centers in Virginia 


July 
1992 


March 
through 
May 
1992 


Develop monitoring mechanism for 
group of 15 students from Granby 
HS who did not have 
comprehensive vocational 
evaluation and the control group 


August & 
September 
1992 


August 
1992 


Develop survey to be used with 
special education faculty 


August 
1992 


August 
1992 


Develop survey to be used with 
secondary special education 
students 


August 
1992 


August 
1992 
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Conduct survey of secondary 
special education teachers 


September 
1992 


October 
1992 


Conduct interviews of 
special education students 


September 
1992 


March 
1993 


Consult with vocational 
evaluators throughout VA on 
results of survey and proposed 
innovations 


October 
1992 


October 
1992 


Meet with vocational evaluators 
to discuss the statewide input 


October 
1992 


October 
1992 


Develop and implement the 
innovations 


October 
1992 


August 
and 
September 
1992 


Conduct inservice activity for 
special vocational programs 
staff on new methods, 
activities, strategies for 
imp 1 ement at ion 


October 
1992 


October 
1992 


Conduct evaluation of the above 
inservice activity using a 
formal evaluation instrument 


October 
1992 


October 
1992 


Conduct inservice training on 
new innovations in vocational 
evaluation with administrators, 
special educators, vocational 
educators, vocational resource 
teachers, and guidance 
counselors 


October 
1992 


Small 
groups 
throughout 
1992-1993 


Conduct vocational evaluations 
using both the innovations and 
the previous comprehensive 
techniques 


November 
1992 


September 
1992 


Collaborate with coordinator of 
special education to determine 
methods for evaluating the 
innovations in conjunction with 
special education teachers 


November 
1992 


November 
1992 


Continuously identify and 
develop methods for the 
evaluation of results of the new 
innovations 


November 
through 
June 
1993 


June 
1993 



ueveiu^ lueunua ox laenuiiying 
special needs students in grades 
eight through twelve 


February 
1993 


February 
1993 


the Student Information System 
(SIS) 


December 
1992 


March 
1993 


Administer Differential Aptitude 
4. t.o lu riiyiiun tjraaers city wide 


December 
1992 


December 
1992 


Interpret results of 
Differential Aptitude Testing 

w DLUUciiUo du an lUldClie 

schools 


February 
through 
May 
1993 


January - 
May 

1993 


Monitor progress in vocational 
programs of Granby HS students 
mi^t £3 uuutciiuo xii uiiti control 
group 


1992- 93 
and 

1993- 94 


September 
through 
June 
1993 


uouawiiau L^vJlUIll J. u L£6 tO COn&UCtl 

follow-up interviews with 
students and faculty on new 
methods of vocational evaluation 


April 
1993 


March 
1993 


Collect and review data on the 
innovations 


May 
1993 


May 
1993 


MSfit With \/ A &\fZ* 1 naf nve< ra4- rt^^^As 

conference and present report on 
efforts and findings to date 


May 1993 


April 1993 


Provide forum for oral and 

Written f PPdha olr f rnm r^-f- K _ ^ ^ 

^ i«» wi 4 a, ccuijauA, irom ouner area 
evaluators on the presentation 


May 1993 


April 
1993 


Meet With NPS ev» 1 npfnrc 

review, revise and document 
first year efforts and develop 
plans for next school year 


May - 

June 

1993 


June 
1993 


Review feedback and include in 
development of final model 


July 
1993 


July 
1993 


Develop a written model of 
Vocational Evaluation Services 
in Norfolk, VA 


April - 
July 
1993 


April 
1993 
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Disseminate model through staff 
development sessions with 
vocational educators, vocational 
resource teachers , special 
educators, guidance counselors, 
and administrators 


April and 
May 

19Q3 

X Z7 Z? -J 


April and 
May 1993 


Develop statewide clearinghouse 
of surplus evaluation materials 


June 
1993 


July 
1993 


Develop final presentation for 
statewide conference on the 
results of the innovations and 
the new Norfolk model of 
vocational evaluation 


July 
1993 


July 
1993 
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Appendix B 
Norfolk Public Schools 
1991-1992 Special Education 
High School Enrollments 
According to Age and Exceptionality 



AGE 


14 


15 


16 


17 


18 


19 


20 




Educable 
Mentally 
Retarded 


20 


28 


35 


22 


9 


11 


5 




Trainable 

Mentally 

Retarded 


8 


4 


6 


4 


6 


7 


14 


8 


Learning 
Disabled 


176 


140 


148 


126 


90 


36 


8 


1 


Emotionally 
Disturbed 


40 


40 


24 


26 


13 


8 


5 


o 


Visually 
Impaired 


1 


6 


0 


0 


2 


0 


0 


0 


nearmg 
Impaired 


3 


4 


2 


7 


3 


0 


0 


0 


' Speech/ 
Language 


11 


4 


5 


3 


1 


1 


0 


0 


Multiply 
Handicapped 


3 


4 


3 


2 


1 


1 


0 


0 


Orthopedic 
Impaired 


1 


0 


4 


1 


0 


0 


0 


0 


Other 

Health 

Impaired 


1 


1 


1 


2 


4 


1 


0 


0 


Severe/ 

Profound 

Handicapped 


8 


2 


3 


8 


7 


6 


6 


4 
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Appendix C 

Survey of Other Vocational Evaluation Programs in 
School Divisions in Virginia 



Dear Vocational Evaluator: 

I am conducting research on the methods and processes 
being used across the state for completing vocational 
evaluations of special education students in school-based 
settings. As you are probably aware, I have been involved 
in the vocational evaluation of special populations in 
Virginia since 1973, The processes and methods that we 
developed to use in the state's rehabilitation settings were 
carried over into school-based assessments when school 
centers began opening in the late 1970s . 

As I have visited school-based evaluation centers, I 
have recognized numerous similarities to that original 
model. There have been numerous changes in technology and 
in most employers' expectations over the past 20 years. 
School vocational programs for special populations have 
changed immensely during that time period. Even our 
enabling legislation has been through several major 
evolutions during those years, 

I am trying to determine how evaluation centers have 
modified the original processes and methods and the current 
school-based vocational evaluation profile for Virginia, 
Please take time to respond to the attached questionnaire 
and return it in the enclosed envelope. Please attach 
descriptions of any other innovations being used in your 
setting. If you would like to receive a composite of these 
results, please attach a note to that effect including the 
mailing address to which they should be sent, 

I appreciate your interest in vocational evaluation and 
your assistance in responding to this survey. If you have 
questions, please contact me at (804) 441-2957, 

Sincerely, 



Glenda D. Feldt, CVE, CWA 

Program Leader, Vocational Education 

Programs for Special Populations 
Attachments (2) 

117 



ERIC 



12\> 



A SURVEY OF TRENDS IN SCHOOL-BASED 
VOCATIONAL EVALUATION IN VIRGINIA 

Please identify yourself and worksite on the questionnaire. 
This will help with survey follow-up. Data will be used in 
composite format only, without any identification of 
individuals or school districts. 



Name 



Work address 



School district total enrollment 



Special education secondary enrollment 

Is your district rural? urban? 

suburban? 

Number of evaluators on your staff 



Full-time secretary? part-time? 

none? 



Are narrative reports completed on each student? 



Is a checklist report completed instead? 



Where did you get the report format you use? 



Are you willing to share your format with others? 



How many students per evaluator per week? 

Average number of days students in evaluation 

Do you have at least one "no show" per week? 

Do you have more than one "no show" per week? 

How many alternates do you schedule per week? 

How many completed vocational evaluations last year? 

How many students exited the evaluation before completinq 
their vocational evaluation? 
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What percentage of the students evaluated were special 
education? disadvantaged? other? 



THE PROCESS 

How are special education students referred to you? 



Do you receive referral packets prior to the attendance of 
the students? 

What is included in your referral packet? 



Do you usually get all of the referral information you need? 



Who recruits students for the. evaluation? 



Do you use a brochure in this initial recruitment? 



Do you use a videotape in this initial recruitment? 



Do you visit/observe students in their academic classrooms 
either prior to or during the vocational evaluation? 



Do you visit/observe students in their vocational classrooms 
either prior to or during the vocational evaluation? 



Do you personally access the individual student's cumulative 
or confidential school records? 



Do you collect any written data on student performance from 
vocational teachers prior to evaluations? 

Does your school division participate in PERT? 

If yes, how many students did they evaluate from your 

district last year? the year before? 

Are you involved in the selection of students for PERT? 
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Does your school division receive vocational evaluations 
through the Department of Rehabilitative Services (other 
than PERT) on students while they are still enrolled in 
school? 

What is your involvement in that process? 

THE METHODS 

Do you use the JEVS Work samples? 

If yes, do you use their norms? 

If you use other norms, how were they derived? 



Do you use VALPAR Work samples? 

If yes, do you use their norms? 

If you use other norms, how were they derived? 



Do you use Singer Work samples? 

If yes, do you use their norms? 

If you use other norms, how were they derived? 



Do you use Choice Work samples? 

Have you rewritten the administration/ instructions? 
What norms do you use? 



How were they derived? 



Do you use Tower or Microtower Work samples? 
If yes, do you use their norms? 
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If you use other norms, how were they derived? 



Do you use MECA Work samples? 

If yes, do you use their normsV 

If you use other norms, how were they derived? 



Do you use VIEWS Work samples? 

If yes, do you use their norms? 

If you use other norms, how were they derived? 



Do you use McCarron-Dial System? 

If yes, do you use their norms? 

If you use other norms, how were they derived? 



What other commercially developed work samples do you use? 



What commercially developed work samples do you have that 
are not used? 



Are you interested in selling any of these? 

Are you interested in sharing any of these? 

Do you utilize ASVAB data? 

Do you utilize GATB data? 

What work samples have you developed (homemade)? 



Do they relate to a specific local business/ industry? 
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If yes, which businesses or industries? 



How do you validate your locally developed work samples? 



Have you developed industrial norms for any of vour 
work samples? 1 * 

How frequently have you used situational assessments? 

e?aluated? dUCt follow * u P studies on students you have 

If yes, have you developed the data into an outcomes report? 

Are you willing to share this? 

Have you had the vocational instructors actually complete 
the worksamples related to their teaching areas? ipj - ete 

Do the vocational instructors provide you with informal 
feedbaw on the students who were recommended for their 
programs? 

How do you use this information? 



Do you sit in on student IEP conferences? 



Transition conferences? child study meetings? 
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Appendix D 



A Tally of Surveys Conducted with School-Based 

Vocational Evaluation Programs in Virginia 

Responses were solicited from all Virginia school divisions, 
first by post card to the superintendent. Every school 
division responded to the post card mailing (100% response 
from superintendents) . This was followed by presentation of 
this survey to those districts who reported that they have 
school-based vocational evaluation programs (n = 15) . All 
responses are reported in composite form so that individual 
school divisions are not identifiable. 

Fifteen districts or regional evaluation centers received 
surveys. Thirteen responded to the questionnaire. One 
division reported that they were prohibited from releasing 
any information outside of their school district due to a 
local school board policy and some current litigation 
issues. The response rate of completed questionnaires was 
87%, a phenomenal return rate, which may reflect a high 
level of interest across the state in developing revisions 
in the process of vocational evaluation. 

*************************** *************** 

School district total enrollment Ranged from 10,188 to 

78,760 . 

Special education secondary enrollment Ranged from 289 to 

6,000 

Is your district rural? 3 urban? 4 suburban? 6 



Number of evaluators on your staff 
Rural = 1 per system 
Urban = 2 per system 

Suburban = ranged from 1 to 3 per system, with no 
correlation between special education 
enrollment or total enrollment and number 
of evaluators. (One very large suburban 
district has one evaluator while the 
smallest reporting suburban district has 
two evaluators.) 

Do you have a secretary: 

Full-time? Rural = 1; Urban 
Part-time? Rural = 1; Urban 
None? Rural = 1; Urban 
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= 2; Suburban = 2 
= 1; Suburban = 3 
= 1; Suburban = 1 



Are narrative reports completed on each student? 

Yes = 12; No = 1 ("Composite checklist is provided because 
I only conduct Apticom testing. 11 ) One respondent added a 
note that checklists are provided in lieu of reports for 
students completing a Phase I assessment. 

Is a checklist report completed instead? See above 

Where did you get the report format you use? 

From original reports sed in Virginia since 1978: 11 

Created own format: 1 

From format used in West Virginia: 1 

Are you willing to share your format with others? 

Yes = 12 ; No response = 1 

How many students per evaluator per week? 

Phase I: Rural -=2 0 
Phase I: Urban = 0 
Phase I: Suburban = 6-3 6 
Comprehensive: Rural =2-5 
Comprehensive: Urban = 3-8 
Comprehensive: Suburban = 2-3 

Average number of days students in evaluation 

Phase I: 4 5 mins to 4 hours 
Comprehensive: 3 to 4 days 

Do you have at least one "no show 11 per week? 
Yes =7: No = 6 

Do you have more than one "no show" per week? 
Yes = 6: No » 7 

How meiny alternates do you schedule per week? 
Zero = 8: Two = 5 
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How many completed vocational evaluations last 





One 

Evaluator 


Two 

Evaluators 


Three 

Evaluators 


Rural 


3 
112 
115 






Urban 




170 
259 
190 
126 




Suburban 


0 
112 


165 
186 


272 
400 



How many students exited the evaluation before 
completing? 



Rural responses: 0% (of 3 students); 3%; 5% 
Urban responses: 1%; 8%; 9%; io% 

fnew UbUrban responses: Not applicable on one response 

respondents); 5%; 7% Cent2r) 2%; 4% < 2 

SS~E ?rCent 5? e ^ f thS students evaluated were special 
education, disadvantaged, other? 

(Responses written in as "other" were descriptors of 
disadvantaged populations, using the 
Department of Education's definition.) 





Special Education 


disadvantaged 


Rural 


10% 


90% 




50% 


50% 




50% 


50% 


Urban 


60% 


40% 




61% 


39% 




97% 


3% 




55% 


45% 


Suburban 


3 0% 


70% 




37% 


63% 




82% 


18% 




48% 


52% 




73% 


27% 




26% 


74% 
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THE PROCESS 



How are special education students referred to you? 

By Special Education Teachers: 9 

By Vocational Resource Teachers: 3 
By Guidance Counselors: 7 

(The most frequent response was a combination of 
special education teachers and guidance 

counselors; most school divisions had more than 

one source of referrals.) 

Do you receive referral packets prior to the attendance of 
the students? 

Yes = 11; Sometimes = 3 



What is included in your referral packet? 

This varied widely across the responses. The materials 
included and number of divisions reporting 
positively on that item are: 

Referral Form: 11 
Psychological Evaluation: 6 
Parent Permission: 10 
Transcript: 7 
Medical Evaluation: 4 
Achievement Test Scores: 5 
Copy of IEP: 2 
Attendance Records: 3 
Report Cards: 2 
Behavior Checklists: 3 

Do you usually get all of the referral information you need? 

Yes =7; No = 6 

Who recruits students for the evaluation? 

Special Education Teachers: 6 
Vocational Resource Teachers: 2 
Guidance Counselors: 3 
Vocational Evaluators: 5 

Do you use a brochure in this initial recruitment? 



Yes =6; No = 7 
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Do you use a videotape in this initial recruitment? 

Yes = 3 (Slides = 1) 
No = 9 

Do you visit/observe students in their academic classrooms 
either prior to or during the vocational evaluation? 

Yes =0; No = 13 

Do you visit/observe students in their vocational classrooms 
either prior to or during the vocational evaluation? 

Yes = 0; Sometimes =3; No = 10 

Do you personally access the individual student f s cumulative 
or confidential school records? 

Yes =3; No = 10 

Do you collect any written data on student performance from 
vocational teachers prior to evaluations? 

Yes = 3 ; No = 10 

Does your school division participate in PERT? 

Yes =6; No = 7 
If yes, how many students did they evaluate from your 
district last year? Ranged from 2-30 
the year before? Ranged from 8-30 

Are you involved in the selection of students for PERT? 

No school district evaluators were on the selection 
teams , 



Does your school division receive vocational evaluations 

through the Department of Rehabilitative Services (other 
than PERT) on students while they are still enrolled in 
school? Yes =3; No = 10 

What is your involvement in that process? None = 100% 



THE METHODS 



Ex planation of Reporting Format 

Respondents were asked to indicate which commercially 
developed work sample systems they use in their vocational 
evaluation centers. These are summarized in table format, 
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including reported use of commercially developed 
(industrial) norms or use of local norms which compare 
students to other students. If a work sample was utilized 
as an exploratory activity without norm application, that is 
also noted. 

Finally, respondents were asked to write in any additional 
commercially developed work samples they use. They were not 
asked to provide normative information on the write-in 
responses . 

These are summarized in the table on the following page. In 
addition, the respondents were asked to identify unused 
items. 

***************************** ******** 

What commercially developed work samples do you have that 
are 

not used? 



None = 1 TAP = 2 SAGE = 2 

Singer = 3 VASCO = 1 Prep-Coats = 2 

Valpar = 3 VIEWS = 1 

Choice » 4 Mesa = 1 



Are you interested in selling any of these? Positive 
responses, except most ^valuators felt they would probably 
not be allowed to sell because items were purchased with 
special funds. 



Are you interested in sharing any of these? See above 
Worksample Utilization 





Divisions 
Using Item 


Using the 

Commercial 

Norms 


Using Local 
Norms or 
Directions 


Used to 
Explore 
(only) 


JEVS 


3 


1 


2 




VALPAI 


10 


8 


2 




SINGER 


12 


6 


4 


4 


CHOICE 


12 




i ° 


5 


TOWER 


1 


1 






MECA 


3 


3 






VIEWS 


8 


6 


1 


1 
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McCARRON- 
DIAL 


4 


4 






VITAS 


4 


Not avail 






PREP- 

PHYTIC 


4 


Not 

available 






O A TUT 


2 


Not avail 






APT icon 


5 


Not avail 






TRANSIT 


1 


Not avail 






SSSQ 


1 


1 






CRAWFORD 


1 


1 







PENNSYLV. 
BIMANUAL 


1 


1 






BENNETT 
HAND TOOL 




1 






PROJECT 
DISCOVERY 


1 


Not 

available 







Do you utilize ASVAB data? Yes = 3 ; No = 10 



Do you utilize GATB data? Yes =5; No = 8 
What work samples have you developed (homemade)? 



None 4 
Sheetrock Hanging 1 
Door Lock Repair 1 

Graphic Arts 3 

Sewing 1 

Cashier 2 

Cleaner 4 

Masonry l 

Auto Body 1 

Greenhouse 2 

Auto Mechanic 4 

Electricity 3 

CADD 2 

Cosmetology 3 

Cooking 3 



Do they relate to a specific local business/ industry? 
Yes =4; No = 5 

If yes, which businesses or industries? Most respondents 
omitted 
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How do you validate your locally developed work samples? 

ttSnUS 1 ?* 18 1 Not Validated 7 

student Success l 

Have you developed industrial norms for any of your 
work samples? y Y r 

All respondent- g answered : Hn 
How frequently have you used situational assessments? 
Often: 1 Seldom: 5 Never: 7 

D ° Y °eva?Sted? f0ll0W " UP StUdiSS ° n Y°* have 

Yes =8; No = 5 

If yes, have you developed the data into an outcomes report? 

Yes = 5 ; No = 3 
Are you willing to share this? 

Yes =3; No = 2 

Have you had the vocational instructors actually comolete 
the work samples related to their teaching areas? ? 

Yes =1; No = 9; Some of them = 3 

°° th ?eedback°on\h« S ^ U f P ^° Vide you with inf °™al 

programs? st ^ents who were recommended for their 

Yes =7; No = 6 

How do you use this information? 

Planning, program evaluation. 

Three respondents said, "Not at all." 

Do you sit in on student IEP conferences? 
Sometimes = 9 ; No = 3 (i no response) 

Transition conferences? 

Sometimes = 2; No = lo (1 no response) 

Child study meetings? 

Sometimes = 3; No = 9 (i no response) 
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Appendix E 
Addendum to the Norfolk Public Schools 
1992-1993 Testing Schedule 



September 23, 1992 



MEMORANDUM 



TO: Middle School Principals 

Head Counselors 
Eighth-Grade Teachers 



FROM: 



E. P. Antoine, II, Acting Assistant Superintendent for General 

Administration and Personnel 
Margaret B. Saunders, Assistant Superintendent, Instructional 

Support Services ^flUhi 
Shirley B. Wilson* Assistant Superintendent, Secondary Schools 

and Centers iyhiAJ , 
Pamela Kloeppel, Senior Coordinator of Guidance ' 4^ 
A and n stat^ti r> ^^^^^^ Department of Res earch, Testing 
Lillian S. Holloway, Senior Coordinator, Testing Programs 




COPIES: Mr. John F. Smith-, Sr. 
Mr. Frank Peele 
v/Mrs. Glenda Feldt 

SUBJECT: ADDENDUM TO THE 1992-93 TESTING SCHEDULE 

Beginning this year, the Differential Aptitude Tests (DAT) will be administered to 
eighth grade students in Norfolk Public Schools. This is being done for several 
compelling reasons: 

1. The Tech-Prep program of studies begins in 9th grade, making it 
essential that students have early identification of vocational aptitudes 
and interests to facilitate appropriate course selection. 

2. Federal regulations for vocational education require assessment of 
vocational aptitudes and interests for most students in Norfolk Public 
Schools prior to enrollment in a vocational program. 

3. Federal monitoring will- occur in May, 1993, and we must show that 
assessments are being conducted and career information provided bv 
ninth grade. 9 

?Q h ao F ° m V ' ^ administered to all eighth graders on December 7-18, 

1992. It is recommended that testing be scheduled for approximately 1 hour per day 
over f 1V e days . Planning for the administration of the DAT will be discussedat the 

SCHOOL ADMINISTRATION BUILDING, POST OFFICE BOX 13S7. NORFOLK. vwr.iMiA 23S01 



131 

133 



Memorandum 

Addendum to the 1992-93 

Testing Schedule 
September 23, 1992 
Page 2 



scheduled Middle School Head Counselors* meeting on November 18, 1992. 
Thereafter, the eighth grade cluster leaders will be inserviced by the head 
counselors . 

A suggested testing schedule and & DAT manual are enclosed. for your information. 
If you have any questions or concerns, please contact Lillian S. HoHoway at 441- 



2319. 



blw 



Enclosure 
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A Suggested Testing Schedule 

F>re-DaY Testing 

The total administration time for the OAT and CAREER Planning 
Questionnaire is about fo«r hours and 40 minutes. 



Gr. 8 - Form V 



DAT TESTING TIMES 



ACTUAL APPROXIMATE APPROXIMATE 

TESTING ADKINISTRAXIOK TOTAL 

TIKE (KTNOTES) TIME (MINUTES) TIME (MINUTES) 



Day X 



Introduction 

Career Planning Questionnaire 
Answer Sheet Preparation (DAT) 



35 
20 



55 min. 



Day 2 

Verbal Reasoning 
Spelling 



30 
10 



8 
S 



38 
IS 



S3 min. 



Day 3 

Numerical Ability 
Abstract Reasoning 



30 
20 



35 
2S 



GO BrLru 



Day 4 

Clerical Speed and Accuracy 

Part 1 3 
Part 2 3 

Mechanical Reasoning 30 



8 
5 

38 



51 inin. 



Day S 

Space Relations 
Language Osage 



2S 
20 



33 
2S 



58 sain. 



4 hrs. 37 min. 
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Appendix F 
Norfolk Public Schools 
Middle School Vocational Assessment Model 



Data Collection 




Development of 
Vocational Plan 




Curriculum-Based 

Vocational 

Assessment 






i 


i 



Differential 
Aptitude Test (DAT) 



IDEAS or other 
Vocational Interest 
Inventory 



Recommendations 
for High School 
Programs 



Careers fc You 
Exploratory 

Class 
Teacher Report 



Interpretation of 
Results to Students 
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Appendix. G 



NORFOLK PUBLIC SCHOOLS 
HIGH SCHOOL VOCATIONAL ASSESSMENT MODEL 



PHASE ONE fCHOOL BASED 



Student 
Identification 




Data 

Collection 




Development 
of Vocational 
Skills Profile 










J 








Review 
Data on File: 

Educational 

Sociological 

Medical 

Psychological 

Attendance 

Discipline 




Introdu ce 
Student to 
Vocational 
Assessment 













Vocational Planning 
Counseling 



Student Interview 
Teacher Interview 
Interest Inventory 
Behaviors 
Learning Styles 
Classroom Observation 
Aptitude Testing 
Curriculum Baaed 
Vocational Assessment 



Recommendations : 

1. Tech Prep or Vocational 
Education 



a. 


Health £ 




Human Svcs. 


b. 


Business £ 




Marketing Svcs. 


c. 


Engineering £ 




Technical Svcs. 


d. 


Fine Arts 


e. 


EFE/WECEP 


f . 


Other 



2. 



College Preparatory 
Curriculum 



Synthesis and Interpretation 
of data 



Comprehensive Vocational 
Evaluation (Phase Two) 



Report - Checksheet 



PHASE TWO - CENTER BASED 



phase three * feedback 



Attend Vocational 
Evaluation Center 



Types of Further 
Assessmenti 

Physical Capacity 
Screening 
Work Samples/ 
Vocational Exploration 
( "Hands-On" ) 
Situational Assessment 
Work Behavior 
Psychometric* 
Vocational Interest 
Aptitude Testing 
Achievement Testing 



Synthesis and Interpretation 
of Data/Evaluation Report 



I 



Vocational Planning/Counseling 



Monitoring Student 
Progress 



Review Current 
Records and 
Placement 



Additional and 

Comprehensive 

Evaluation 



Further Rao 



ndation 



Other Classes 

Counseling 
Other Services 
Accommodations 



135 



1 i 3 

BEST COPY AVAILABLE 



Appendix H 

Vocational Evaluation Schedule for 1992-1993 
Indicating Potential Numbers of Students to be Served 
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Appendix I 

Population Analysis: Norfolk Public Schools 
Vocational Evaluation Center 
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Appendix J 

Differential Aptitude Test (DAT) Summary 



NORFOLK PUBLIC SCHOOLS .SCHOOL VEAR 1992 - joo? 

VOCATIONAL EVALUATION CENTER 



Differential Aptitude Test Summary (DAT) 

Students Served 



REFERRAL SOURCE 


NUMBER OF STUDENTS TESTED 


School 


Oltadvantaqad 


Handlcaopod 


L.E.P. 


TOTALS 


Cranby Klgh 


230 


Mot Availabla 


4 


234 


Laka Taylor Hlqh 


204 


Mot Availabla 


0 


204 


Maury Hlqh 


207 


Mot Availabla 


0 


207 


Morvlaw Hlqh 


231 


Mot Availabla 


0 


231 


Washington Hlqh 


234 


♦"Mot Availabla 


0 


234 


tarry Robinson 


0 


0 


0 


0 


■allantlna 


0 


0 


0 


0 


Transition 


0 


0 


0 


0 


Lltaraey r»« sport 


44 


0 


0 


44 


Axalaa Mlddla 


185 


18 


0 


207 


■lair KLddla 


U« 


IS 


0 


131 


Korthalda Mlddla 


145 


17 


0 


1(2 


Morvlow Mlddla 


206 


21 


1 


230 


Roiwont Middia 


13t 


12 


1 


151 


fturfnar Mlddla 


134 


12 


0 


14 C 


Laka Taylor Mlddla 


115 


25 


0 


140 


Larayatta-WLnona Mlddla 


1*4 


24 


0 


its 


TOTALS 


2,35* 


144 

(Mlddla School Only) 


i 


2,501 



Total *tud«nts Test** witi DAT: 

Cr*d« t 2, ost 

Qradai XQ 1.333 
3,311 
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Appendix K 



Survey of Secondary Special Education Teachers 

Dear Special Educator : 

I am conducting research through Norfolk Public Schools 
related to the process and utilization of vocational 
evaluation for special needs students. Efforts are underway 
to develop methods of evaluating more students in less time 
by using varied assessment techniques. Your input is vital 
to this redesign. Will you please complete this brief 
questionnaire and return it to me by November 30, 1992? 
Your assistance is greatly appreciated. 

Glenda Feldt, Program Leader 
Department of Adult and Vocational 
Education 



Place a mark under the response that best describes your 
opinion. 





S A 


A 


Don't 
Know 


D A 


S D A 


I am familiar with the four- 
day comprehensive vocational 
evaluation. 












I have referred students for 
vocational evaluation. 












I have read a completed 
vocational evaluation report. 












I have interpreted a 
vocational evaluation report 
to a student or parent. 












I have used the vocational 
evaluation in writing IEP's. 












I have used the vocational 
evaluation in determining 
which classes my students 
will take. 












I have visited the vocational 
evaluation center at Madison. 
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I have viewed the orientation 
to vocational evaluation 
videotape which is shown to 

ill jf O ^ \Jk. \jL C X i LO • 












Vocational evaluation is 

pyi f 1 pal <p v» n 1 ann i nrf t»i i tn x r 

ui iLiuax iui jj-Lcnni-Liiy wx my 

students. 












My students seem to enjoy the 

VOL/aulUIlal tlValUaUlun . 












Vocational evaluation takes 












My students can afford to 

mi cc ^ e~\Y* A H^VQ <rf place +- n 

attend vocational evaluation* 












I read all sections of a 6 to 

O JJciytJ t: V d X lid U ±KJl 1 JL c[JUI u • 












I read the recommendations 
part of the report only. 












I have asked the vocational 
evaluators to attend 
staff ings or IEP meetings. 












I have used the evaluation 
report when developing the 
student's transition plcn. 












Mainstreamed students should 
go to Madison for vocational 
evaluation. 












Self-contained students need 
the comprehensive vocational 
evaluation. 












It seems to me that a shorter 
process could be developed 
for mainstreamed students. 












It seems to me that self- 
contained students should be 
able to select their self- 
contained vocational class 
based only on interests. 












An in-building assessment 
procedure is needed. 
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A pencil and paper test will 
reflect my students 1 
vocational aptitudes . 












Self-contained students need 
to explore vocational classes 
before selecting one . 












Success in a vocational class 
at the home school is a good 
indicator of potential for 
higher level courses in 
vocational ed. 













Appendix L 

Tally of Results of a Survey of Secondary 

Special Education Teachers 

SA=Strongly Agree A=Agree DK=Don 1 t Know D=Disagree 

SD=Strongly Disagree 





S A 


A 


D K 


D 


S D 


I am familiar with the four- 
day comprehensive vocational 
evaluation. 


14 


14 




2 




I have referred students for 
vocational evaluation . 


22 


7 






1 


I have read a completed 

vuuaLiuiiai cvaiua tiuil Icpui U* 


O A 


c 
D 




1 




I have interpreted a 
vocational evaluation report 


15 


8 




3 


3 

(1= 
nr) 


I have used the vocational 

PV/iliiat'lon in ur t +* i nrr • o 

CValua L.1UJI J.J1 Wl XUlll^ XC*IT O. 




J.O 




1 


l 


I have user* the vocational 
evaluation in determining 
which classes my students 
will take. 


14 


12 


1 


1 


o 


I have visited the vocational 
evaluation center at Madison. 


7 


1 




7 


15 


I have viewed the orientation 
to vocational evaluation 
videotape which is shown to 
my students. 


11 


6 


2 


3 


8 


Vocational evaluation is 
critical for planning with my 
students . 


16 


10 




3 


1 


My students seem to enjoy the 
vocational evaluation. 


5 


12 


8 


3 


2 


Vocational evaluation takes 
too long. 


4 


9 


6 


6 


5 
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My students can afford to 
iuiss j or ** uays or ciass to 
attend vocational evaluation. 


Q 
O 


/ 


z 


b 


-7 

7 


I read all sections of a 6 to 
o page evaluation report . 


Q 


11 




/ 


3 


I read the recommendations 
part of the report only. 


2 


3 




14 


11 


I have asked the vocational 
evaluators to attend 
staff ings or IEP meetings. 


2 




1 


13 


14 


I have used the evaluation 
report when developing the 
student's transition plan. 


11 


16 




2 




Mainstreamed students should 
go to Madison for vocational 
evaluation. 


7 


5 


8 


3 


6 


Self-contained students need 
the comprehensive vocational 
evaluation. 


16 


8 


4 


2 




It seems to me that a shorter 
process could be developed 
for mainstreamed students. 


12 


7 


10 




1 


It seems to me that self- 
contained students should be 
able to select their self- 
contained vocational class 
Dasea oniy on interests • 


3 


8 


5 


9 


5 


An in-building assessment 
procedure is needed . 


11 


6 


9 


3 


1 


A pencil and paper test will 
reflect my students' 
vocational aptitudes . 




3 




11 


16 


Self-contained students need 
to explore vocational classes 
before selecting one . 


12 


15 


1 


2 




Success in a vocational class 
at the home school is a good 
indicator of potential for 
higher level courses in 
vocational ed . 


8 


12 


7 


1 


2 
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Number of Surveys Distributed = 64 
Number of Surveys Returned =34 
Total Response Rate =53% 

Special Note: 4 of the returned survey forms were left 
blank with notations that the teachers were out on long term 
leave. 
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Appendix M 



Brochure Describing Vocational Assessment Services in 
Norfolk Public Schools 



VOCATIONAL ASSESSMENT 
REPORTS 

A copy of the DAT profile sheet is placed in 
each student's school file. In addition, 
students who participate in CBVA and Phase 
I Assess ments will have assessment 
information on-line through the Student 
Information System (SIS). A written report 
will be provided for students who complete 
a Phase if comprehensive evaluation. 

INFORMATION 

For additional information on vocational 
assessment and evaluation, talk with your 
school guidance counselor, vocational 
teacher, special education teacher or 
telephone the Department of Adult and 
Vocational Education at (804) 441-2957. 



SCHOOL BOARD 
OF 

THE CITY OF NORFOLK 
Dr. Lucy R. Wilson, Chairwoman 

Rev. G. Wesley Hardy, 
Vice-Chairman 

Elizabeth C. Parkman 

Anita O. Poston 

Ulysses Turner 
Joseph T. Waldo 
Dr. Robert F. Williams 



SUPEMNTENDENT OF 
SCHOOLS 

Dr. Roy D. Nichols, Jr. 




VOCATIONAL 
ASSESSMENT AND 
EVALUATION 




BELIEVE-AC33IEVE-SUCCEED! 
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COAJL OK VOCATIONAL 
ASSESSMENT 

Norfolk Public Schools offers a variety of 
vocational education options for secondary 
students. These include: 

• Tech-Prep programs in four career 
clusters 

• Pre-vocational and vocational 
electives 

• Occupational courses at two 
vocational centers 

• Apprenticeship options 

Vocational assessment is offered to assist 
students in selecting vocational education 
studies. 

VOCATIONAL ASSESSMENT 

Norfolk Public Schools provides a range of 
vocational assessment activities. To help 
students plan their high school course of 
studies, vocational assessment activities 
begin .in the eighth grade. Most students 
will participate in the Differential Aptitude 
Test (DAT) and will complete a vocational 
interest inventory. This information will 
provide a picture of each student's career 
interests and abilities. Vocational evaluators 
and counselors will interpret these results 
with students and a copy of the profile will 
be given to students. 

CURMCUIAJM-BASED AND 

PHASE I 
VOCATIONAL ASSESSMENTS 

Some students will have additional support 
in vocational planning through Curriculum- 
Based Vocational Assessment (CBVA) in 
middle or high school. From classwork 
completed in the vocational education 
classes, additional career options will be 
suggested. Some students will have Phase I 
assessment in ninth grade. It involves 
reviewi ng student records , interviewing 
students and reviewing performance in 



vocational classes. Siudcnib will receive 
more information on career options which 
they may want to pursue in high school. 

PHASE H - COMPREHENSIVE 
VOCATIONAL EVALUATION 

Certain students will have the opportunity to 
attend the Vocational Evaluation Center for 
one or more days for "hands-on" career 
exploration and assessment. Students will 
actually try out a variety of jobs such as: 

Air Coodiuoaing/Re/rigeration 

Assembly Work 

Auto Body Repair 

Auto Mechanics 

Auto Servicing 

Business/Office 

Carpentry /Coast ruction 

Cashier 

Child Care 

Drafting 

Electrical 

Electronics 

Food Service/Catering 

Furniture Refinishing 

Grapoic/Coaunercial Art 

Home Health/Health Careers 

Horticulture 

Housekeeping 

Laundry 

Marketing/Sales 

Masonry 

Plumbing 

Public Safety 

Sewing/FcshioQ 

Warehousing 

Welding/Shipbuilding 

Students will learn about their work 
potential from these activities. They will 
receive suggestions for vocational training to 
assist in planning for high school and 
postsecondary occupational training. 

To attend the Vocational Evaluation Center 
students will ride school buses from their 
high schools to Madison Career Center. 
While at the Center students may be asked 
to wear dust masks, aprons, or safety 
goggles for protection while working. 



146 

156 



Appendix N 



Phase I Vocational Assessment Guidelines and Process 
Guidelines 

These are the guidelines to be followed when arranging Phase 
I vocational assessments for students at the high school. 
This information is a supplement to the Norfolk Public 
Schools Vocational Evaluation Operations Manual. 

1. Refer students who are freshmen and sophomores and who 
are mainstreamed or in coteaching classes. 

2. Ref^r students who are appropriate for vocational 
training. Discuss other individual cases with the 
Program Leader. 

3. The method of referral is simple. Provide a list of 
students 1 names and Social Security numbers to the 
vocational evaluators. They will schedule dates to 
visit your school and inform you. 

4. Before the evaluators arrive, notify the faculty at 
your school that certain students will have to be 
released from class for up to one hour on the 
assigned date. Students should not be prohibited from 
meeting with the evaluator. 

5. When the evaluators arrive you should provide the 
following: 

A. A bell schedule for your school 

B. A map of the school showing fire exits, bathrooms, 
and lunchroom 

C. A copy of each student's schedule or access to the 
master book containing all schedules 

D. Access to cumulative and confidential folders 

E. A place to work with the files and to interview 
students and teachers. 

6. After the evaluators have visited, they will return a 
brief Phase I assessment report on each student. These 
recommendations may be used for placement just as a 
comprehensive report is used. 

7. If the Phase I assessment recommends a Phase II 
(comprehensive) vocational evaluation, then all you 
need to send as the referral packet is the parent 
permission form . 
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8. Students who will participate in a Phase II 

(comprehensive) vocational evaluation should be shown 
the videotaped orientation before they go to Madison. 



Noifolk Public Schools, October, 1992 
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Procedures for Phase I Vocational Assessments 



I. Preparation at school site prior to evaluator's visit 

A. Records access and work location 

B. List of students to be evaluated 

C. Provide bell schedule, map of school 

D. Provide teachers 1 , students 1 schedules 

II. Data collection at school site 

A. Records review to include: 
Psychological 
Educational 

Medical 

Social history 

IEP - % of time mainstreamed 

Credit sheet/class schedule 

Grades/report card 

Disciplinary 

Attend: nce/punctuality 

B. Student interview 

Use form to ascertain background, interests 
Previous vocational evaluation results 
Previous vocational classes 

C. Teacher interview 

Use CBVA teacher questionnaire form 
Classroom behaviors 
Overall performance 
Opinion of future potential 

D. Parent interview 
Expectations 

Other considerations 

III. Report recommendations to the resource teacher and 
to the IEP manager 

A. Give immediate feedback (carbon pack form) 

B. Discussion of results 

IV. Submit written report to referral source at school 
site 

V. Enter results into Student Information System (SIS) 



Norfolk Public Schools, October, 1992 
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Memorandum Describing Procedures for Interpretation of 
Differential Aptitude Test Results with Eighth Grade Students 

January 27. 1993 



MEMORANDUM 



FROM: 



COPY: 




TO: Middle School Prvicipals 

Madison Career Center Principal 
Head Counselors 

E. P. Antoine, 17, Acting Assistant Superintendent for General 

Administration and Personnel 
Aaron A. Gay, JrHirector, Department cf Researdu Testing 
and Statistics 

Lillian S. HoUoway. Senior Coordinator, Testing Programs 

All Assistant Superintendents 
Mr. JoJin F. Smith Sr. 
Mr. Frank M. Peele 
Dr. Pamela C. Kheppel 
Mrs. Glenda Feldt 

SUBJECT : DIFFERENTIAL APTITUDE TEST (DAT) RESULTS - FALL 1992 

Accompanying Has memorandum are the results of the Differential Aptitude Tests whidi were 
administered to your 8th grade students in December. 

Enclosed are 2 copies of the Student Career Planning Report. Representatives from the 
Department of Adult and Vocational Education wU visit your school to interpret the DAT results 
tc your students. Head counselors are asked to call Mrs. FeJdt at 441-2957 to set up 

If you have any ouestions. please call Mrs. Lillian S.'Holloway at 441-2319. 



blw 

Enclosures 



SCHOOL ADMIN ISTRATION BUILDING. POST Off l it BOX 13S7, NORFOLK. VIRGINIA ,i<ni 
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Appendix P 



Schedule for Interpretation of Differential Aptitude Tests 

with Eighth-Grade Students 
Spring, 1993 

Glenda Feldt, Program Leader with the Department of Adult 
and Vocational Education, will meet with eighth grade 
students at each middle school to explain the results of the 
Differential Aptitude Tests. The dates for each school are 
as follows: 



School 



Dates 



Status 



Azalea Middle 

Blair Middle 

Lafayette-Winona 
Middle 

Lake Taylor Middle 

Northside Middle 

Norview Middle 

Rosemont Middle 

Ruffner Middle 

Madison Career 
Center (LTP) 



March 2,3 

April 22, 23, 26 

April 19, 20, 21 

April 7, 8 
March 17, 13, 19 
February 2 6 
May 3, 4, 5 
March 8, 9, 10, 11 
March 31 



Completed 
Completed 
Completed 

Completed 
Completed 
Completed 
Completed 
Completed 
Completed 
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Appendix Q 

Student Information System (SIS) Vocational 
Assessment Computer Screen 



Student: 

School: 



Eraluator: 
Recommend at ions : 



Evaluation Date: 



Student No. 

Age: 

Except: 

Type : DAT 
PHI 
COK 



SuaLaass & K*rk«ti*« 



Sag ring * 



Intro to Bum 
Off. T«ch. 
Word Proc. 
Kmy ftoAxd 
Account 
Dmtk Proc. 
l*qmX Oft* 
Hm4. Off. 
Intro to H3ctg. 
CA«b/Check 
fup«r Hgnt. 



rl Des/LAnd _ 
CXmw. TacH. "~ 
Prtn. T«ch. 
Printing 

K/C Mfrlfl. 

Hdg. Trade 
Carpentry 

(HOC) 

(KTVC) 
Slec-trlc 
Elac/Tecb. 
rurn ftefljt. 

(HCC) 

Auto tody 
Auto H«ch. 
Auto S«nr. 
Elac/Kech. 
Keen. TTAd« 
Ha in 

(HOC) 
KATl TTAdA 

Welding 

■ Tecb. Dr*w 
Oom. Art 
DrAft 



Ufa KAOAg. 
Oantal Aat. 
KAdlcAl Aat. 
Pract. Muraa 
Ha. Kaa. Cirt 

(HOC) 
Inatlt. Ser. 

(HOC) 

Co skatology 
public SAfAty 
Faablon Dea lgn 
Catering 

rood S«rv. 

(HOC) 
(KTVC) 



Intro. Art 
Photograph 
Studio Art 
Driving 

CAXAaiCA 

Crafts 
Or AAA 

Hualc 

JourniUia 



OTKR MCOKKgHDATIOKS t 



CZOt 

Atteod coot 1 
»Afa*V. oontt 

Counseling: 

Dept. Aehab. Sort 



AAOTAluatlons 
Ocmm. S«r« tOATdt 
Faych. Opdatai 
HodlCAl OpdAt«I 
Social Service: 
HiUtaryi 
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Appendix R 

Curriculum-based Vocational Assessment (CBVA) 
Teacher Questionnaire 

CURRICULUM-BASED VOCATIONAL ASSESSMENT 
TEACHER QUESTIONNAIRE 



Wc are collecting vocational data on this student: 
Date of Birth: 



Vocational Class: 



Teacher Completing Form: 



Date 



Please indicate below the skills/aptitudes you've observed when working with this student. Write comments on 
the back. 



Strongly 

Acre* Agree 

This student is able to: 

Follow verbal instructions 
Follow written instructions 
Follow a demonstration 
Follow diagrams 
Use mail tools 
Use large tools 
Work with Hands 
Thread a needle 
Measure with a ruler 
Use measuring cups 
Use gauges/the cm oin ft teci 
Organise work/material* 
Work as part of team 
Complete assigned work 
Express thoughts in writing 
Solve problems 
Work accurately 
Demonstrate creativity 
Visualize objects in 3-D 

Lift/carry 5 10 15 20 25 30 35 40 pounds 
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Disagree 



Strongly 
Disagree 
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Based upon my experience, working with this student and observing classroom/vocational performance, I have 
the following comments: 



Thank you for your assistance in developing vocational plans for this student. 
Return completed form to: 

Andrew MacGowanMJease Harris 
Madison Career Center 
1091 W. 37th Street 
Norfolk, 23508 
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Attachment A 



Educational Leadership Appraisal (ELA) 
Final Report 



by 



Glenda D. Feldt 



A Major Applied Research Project Report presented in 
partial fulfillment of the requirements for the degree of 

Doctor of Education 



Original Submission: 7/11/93 Norfolk, VA 23510 
Current Submission: 7/11/93 (o) 804-441-2957 
Implementation Date: 3/1/92 (H) 804-482-4168 

Committee: Advisor, Dr. CM. Achilles 
Reader, Dr. Ron Newell 

Nova Representative, Dr. Charles Faires 
ELA Senior National Lecturer: Dr. Alan Ellis 



Nova University 



ELA Final Report 
Submission #1 . 
Richmond III Cluster 



Program Leader, Vocational 
Education Programs for 
Special Populations 
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O bjectives for Leadership Growth 

Identified leadership strengths of the practicum 
manager had been learned through participation in the 
Educational Leadership Appraisal (ELA) class. These 
included: oral communication skills, written communication 
skills, decisiveness , creativity, initiative, and 
educational perspective. Secondary leadership strengths, 
identified from ELA and applicable to this project, 
included: risk taking, persistence, individual leadership, 
and flexibility. The ELA study area stressed that 
educational leaders must possess an array of leadership 
qualities and be aware of both their strengths and 
weaknesses. By recognizing dimensions in which I was 
strong, I was able to utilize those skills in the 
development of this project and to improve in the dimensions 
where I believed I needed to grow. 
Methods for Achieving Leadership Growth 

My agenda for growth in the development of this project 
focused on the areas in which I rated myself lowest on the 
Educational Leadership Appraisal • These were: political 
behavior, use of delegation, and group leadership. Because 
I was reviewing and revising an established procedure, which 
many professionals across Virginia (and, I learned through 
my research, across the nation) upheld, I had to tread 
lightly. This was a new approach for me; typically, I move 
boldly forward, confident that my interventions are an 
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improvement over previous methods. However, this has often 
led to conflict and delays in convincing others of the 
appropriateness of the methods. As an improvement in the 
dimension of political behavior, I identified leaders in 
Virginia in the area of vocational evaluation, and I 
included them in the development of alternative strategies 
and methods as often as possible. In addition, I made 
personal and telephone contacts to discuss my proposed model 
with national leaders, who have written many recent journal 
articles on similar issues. These personal contacts were 
with individuals in North Carolina, Ohio, Colorado, 
Wisconsin, Texas, Maryland, Washington, D.C., and Indiana. 
I met twice with members of the American Vocational 
Association (AVA) legislative network who were influential 
in getting the legislation passed, which mandated vocational 
assessment. I corresponded with my state senator's office 
as a follow-up to the legislative visit made through the 
1992 Summer Institute, requesting information from the 
rulemaking committees on implementation of the legislation. 
I have worked diligently to keep other professionals 
informed, to allay their fears and feelings of being 
threatened by these new concepts. I have described my model 
as an informal speaker at two statewide meetings of 
vocational evaluators, and I introduced it in the general 
discussion session at the Sixth National Forum on Issues in 
Vocational Evaluation held in Virginia Beach, VA, March 3-5, 
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1993. 

I structured the research of other specific models from 
across the country by delegating responsibility, so that 
each evaluator had an area of responsibility. I involved 
testing personnel, guidance counselors, school 
psychologists, special education and vocational education 
teachers through surveys, implementation techniques, and 
staff development activities to assure that the new model 
included activities that are appropriate for their students. 
I served as a sounding board for ideas from my evaluators 
and vocational resource teachers as well as my peers 
throughout Virginia, and I have attempted to provide honest 
feedback. I have requested personnel in other areas such as 
guidance, testing, and Management Information Systems to 
assist with various aspects of the implementation. They 
have completed their assignments quickly, efficiently, and 
cooperatively and have reported back to me as requested. 
Again, this has been new behavior on my part. In the past, 
I have tended not to request assistance from other 
professionals within the school division, and I have been 
pleasantly surprised by the positive response and the 
improved level of involvement that has resulted. 

Most of all, I have worked diligently not to be a part 
of the problem. This has been a major personal paradigm 
shift because I was so heavily involved in the development 
of the original vocational evaluation model, which has been 
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the standard in Virginia since 1974. Not only had I been a 
loud and strong proponent of the comprehensive assessment 
model, but I had also been a loud and strong opponent (in 
both state and national forums) of newer, more innovative 
methods such as Phase I (records review) evaluations and 
curriculum-based vocational assessment (CBVA) . I have tried 
to be ever conscious of my personal biases, keeping them 
from overshadowing my judgment for the current and future 
needs of Norfolk's special needs populations. 
Results of Educ ational T,eadsrship Growth 

I have achieved growth in the targeted leadership 
dimensions and have practiced my areas of strength 
throughout the development and implementation of this 
project. I was able to involve my vocational evaluators, 
peers, and superiors to analyze the problem and to develop 
an implementation design. Initial project success in 
developing a model to better assess handicapped students led 
to expansion to a model that will serve about 72% of the 
secondary students in Norfolk Public Schools. I proved that 
the research process was helpful in improving my ability to 
analyze an educational problem and to develop an appropriate 
intervention. 

I have been amazed at the level of cooperation I have 
received from a variety of individuals and groups; that 
support has encouraged me and helped me to grow personally. 
I initiated and participated in staff development activities 
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where I shared the new model in a cooperative atmosphere, 
which promoted positive change. My identified strengths in 
oral communication skills and written communication skills 
have proven to be critical in involving individuals and in 
coordinating these efforts into a replicable division-wide 
assessment model. 

Because project implementation and expansion required 
change in procedures used by various individuals and change 
in a system-wide testing procedure, political behavior and 
group leadership have been highlighted. Concerns voiced by 
the vocational evaluators and an initially negative reaction 
from the state professional association for vocational 
evaluators forced me to polish my skills in the area of 
political behavior. Overcoming these initial concerns 
proved to be an indicator of improvement in this ELA 
dimension. 

All ELA dimensions became more focused during times I 
had to deal with the unrest of some individuals and groups. 
However, in retrospect, I see that my personal areas of 
weakness have been strengthened the most. This project has 
been well received on both the local and state levels and I 
have been asked to submit articles to two professional 
journals and a proposal to present at a national conference. 

To implement this project, I had to rely upon many ELA 
leadership dimensions, and I discovered that the instruction 
I had received in this area was useful. This project began 
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as a means to improve services to handicapped students and 
expanded into a replicable, district-wide model for 
vocational assessment of all students with special needs. 

As previously stated , the ELA study area showed me that 
educational leaders must possess an array of leadership 
qualities. The development and implementation of this new 
assessment model has allowed me to utilize numerous 
dimensions as I cgllaborated with testing personnel, 
Management Information Systems personnel, administrators 
(central office and building level) , evaluators, teachers, 
guidance counselors , school psychologists , parents , and 
students. Throughout these efforts, I have shown use of 
these leadership skills, and the targeted ELA dimensions 
have improved. 
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